1
|
Wu Z, Zhang C, Gu X, Duporge I, Hughey LF, Stabach JA, Skidmore AK, Hopcraft JGC, Lee SJ, Atkinson PM, McCauley DJ, Lamprey R, Ngene S, Wang T. Deep learning enables satellite-based monitoring of large populations of terrestrial mammals across heterogeneous landscape. Nat Commun 2023; 14:3072. [PMID: 37244940 DOI: 10.1038/s41467-023-38901-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2022] [Accepted: 05/19/2023] [Indexed: 05/29/2023] Open
Abstract
New satellite remote sensing and machine learning techniques offer untapped possibilities to monitor global biodiversity with unprecedented speed and precision. These efficiencies promise to reveal novel ecological insights at spatial scales which are germane to the management of populations and entire ecosystems. Here, we present a robust transferable deep learning pipeline to automatically locate and count large herds of migratory ungulates (wildebeest and zebra) in the Serengeti-Mara ecosystem using fine-resolution (38-50 cm) satellite imagery. The results achieve accurate detection of nearly 500,000 individuals across thousands of square kilometers and multiple habitat types, with an overall F1-score of 84.75% (Precision: 87.85%, Recall: 81.86%). This research demonstrates the capability of satellite remote sensing and machine learning techniques to automatically and accurately count very large populations of terrestrial mammals across a highly heterogeneous landscape. We also discuss the potential for satellite-derived species detections to advance basic understanding of animal behavior and ecology.
Collapse
Affiliation(s)
- Zijing Wu
- Department of Natural Resources, Faculty of Geo-Information Science and Earth Observation, University of Twente, Enschede, The Netherlands
| | - Ce Zhang
- Lancaster Environment Center, Lancaster University, Lancaster, UK
- UK Centre for Ecology & Hydrology, Lancaster, UK
| | - Xiaowei Gu
- School of Computing, University of Kent, Canterbury, UK
| | - Isla Duporge
- Department of Ecology and Evolutionary Biology, Princeton University, Princeton, NJ, USA
- U.S. Army Research Laboratory, Army Research Office, Durham, NC, USA
- The National Academies of Sciences, Washington, D.C., USA
| | - Lacey F Hughey
- Conservation Ecology Center, Smithsonian National Zoo and Conservation Biology Institute, Front Royal, VA, USA
| | - Jared A Stabach
- Conservation Ecology Center, Smithsonian National Zoo and Conservation Biology Institute, Front Royal, VA, USA
| | - Andrew K Skidmore
- Department of Natural Resources, Faculty of Geo-Information Science and Earth Observation, University of Twente, Enschede, The Netherlands
- School of Natural Sciences, Macquarie University, Sydney, NSW, Australia
| | - J Grant C Hopcraft
- Institute of Biodiversity, Animal Health, and Comparative Medicine, University of Glasgow, Glasgow, UK
| | - Stephen J Lee
- U.S. Army Research Laboratory, Army Research Office, Durham, NC, USA
| | - Peter M Atkinson
- Lancaster Environment Center, Lancaster University, Lancaster, UK
- Geography and Environmental Science, University of Southampton, Southampton, UK
| | - Douglas J McCauley
- Department of Ecology, Evolution and Marine Biology, University of California, Santa Barbara, CA, USA
| | - Richard Lamprey
- Department of Natural Resources, Faculty of Geo-Information Science and Earth Observation, University of Twente, Enschede, The Netherlands
| | - Shadrack Ngene
- Wildlife Research and Training Institute, Naivasha, Kenya
| | - Tiejun Wang
- Department of Natural Resources, Faculty of Geo-Information Science and Earth Observation, University of Twente, Enschede, The Netherlands.
| |
Collapse
|
2
|
Cubaynes HC, Clarke PJ, Goetz KT, Aldrich T, Fretwell PT, Leonard KE, Khan CB. Annotating very high-resolution satellite imagery: A whale case study. MethodsX 2023; 10:102040. [PMID: 36793672 PMCID: PMC9923222 DOI: 10.1016/j.mex.2023.102040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 01/16/2023] [Indexed: 01/26/2023] Open
Abstract
The use of very high-resolution (VHR) optical satellites is gaining momentum in the field of wildlife monitoring, particularly for whales, as this technology is showing potential for monitoring the less studied regions. However, surveying large areas using VHR optical satellite imagery requires the development of automated systems to detect targets. Machine learning approaches require large training datasets of annotated images. Here we propose a standardised workflow to annotate VHR optical satellite imagery using ESRI ArcMap 10.8, and ESRI ArcGIS Pro 2.5., using cetaceans as a case study, to develop AI-ready annotations.•A step-by-step protocol to review VHR optical satellite images and annotate the features of interest.•A step-by-step protocol to create bounding boxes encompassing the features of interest.•A step-by-step guide to clip the satellite image using bounding boxes to create image chips.
Collapse
Affiliation(s)
| | - Penny Joanna Clarke
- British Antarctic Survey, High Cross, Madingley Road, Cambridge, CB3 0ET, United Kingdom.,School of Engineering, The University of Edinburgh, Sanderson Building, Robert Stevenson Road, The King's Buildings, Edinburgh, EH9 3FB, United Kingdom
| | - Kimberly Thea Goetz
- Marine Mammal Laboratory, Alaska Fisheries Science Center, National Marine Fisheries Service, NOAA, Seattle, Washington, United States
| | - Tyler Aldrich
- Northeast Fisheries Science Center, National Marine Fisheries Service, NOAA, Woods Hole, MA, United States
| | - Peter Thomas Fretwell
- British Antarctic Survey, High Cross, Madingley Road, Cambridge, CB3 0ET, United Kingdom
| | - Kathleen Elise Leonard
- Protected Resources Division, Alaska Regional Office, National Marine Fisheries Service, NOAA, Anchorage, AK, United States
| | - Christin Brangwynne Khan
- Northeast Fisheries Science Center, National Marine Fisheries Service, NOAA, Woods Hole, MA, United States
| |
Collapse
|
3
|
Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys. Sci Rep 2023; 13:947. [PMID: 36653478 PMCID: PMC9849265 DOI: 10.1038/s41598-023-28240-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 01/16/2023] [Indexed: 01/19/2023] Open
Abstract
Imagery from drones is becoming common in wildlife research and management, but processing data efficiently remains a challenge. We developed a methodology for training a convolutional neural network model on large-scale mosaic imagery to detect and count caribou (Rangifer tarandus), compare model performance with an experienced observer and a group of naïve observers, and discuss the use of aerial imagery and automated methods for large mammal surveys. Combining images taken at 75 m and 120 m above ground level, a faster region-based convolutional neural network (Faster-RCNN) model was trained in using annotated imagery with the labels: "adult caribou", "calf caribou", and "ghost caribou" (animals moving between images, producing blurring individuals during the photogrammetry processing). Accuracy, precision, and recall of the model were 80%, 90%, and 88%, respectively. Detections between the model and experienced observer were highly correlated (Pearson: 0.96-0.99, P value < 0.05). The model was generally more effective in detecting adults, calves, and ghosts than naïve observers at both altitudes. We also discuss the need to improve consistency of observers' annotations if manual review will be used to train models accurately. Generalization of automated methods for large mammal detections will be necessary for large-scale studies with diverse platforms, airspace restrictions, and sensor capabilities.
Collapse
|
4
|
Moreni M, Theau J, Foucher S. Do you get what you see? Insights of using mAP to select architectures of pretrained neural networks for automated aerial animal detection. PLoS One 2023; 18:e0284449. [PMID: 37093866 PMCID: PMC10124840 DOI: 10.1371/journal.pone.0284449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 04/01/2023] [Indexed: 04/25/2023] Open
Abstract
The vast amount of images generated by aerial imagery in the context of regular wildlife surveys nowadays require automatic processing tools. At the top of the mountain of different methods to automatically detect objects in images reigns deep learning's object detection. The recent focus given to this task has led to an influx of many different architectures of neural networks that are benchmarked against standard datasets like Microsoft's Common Objects in COntext (COCO). Performance on COCO, a large dataset of computer vision images, is given in terms of mean Average Precision (mAP). In this study, we use six pretrained networks to detect red deer from aerial images, three of which have never been used, to our knowledge, in a context of aerial wildlife surveys. We compare their performance along COCO's mAP and a common test metric in animal surveys, the F1-score. We also evaluate how dataset imbalance and background uniformity, two common difficulties in wildlife surveys, impact the performance of our models. Our results show that the mAP is not a reliable metric to select the best model to count animals in aerial images and that a counting-focused metric like the F1-score should be favored instead. Our best overall performance was achieved with Generalized Focal Loss (GFL). It scored the highest along both metrics, combining most accurate counting and localization (with average F1-score of 0.96 and 0.97 and average mAP scores of 0.77 and 0.89 on both datasets respectively) and is therefore very promising for future applications. While both imbalance and background uniformity improved the performance of our models, their combined effect had twice as much impact as the choice of architecture. This finding seems to confirm that the recent data-centric shift in the deep learning field could also lead to performance gains in wildlife surveys.
Collapse
Affiliation(s)
- Mael Moreni
- Department of Applied Geomatics, Université de Sherbrooke, Sherbrooke, Quebec, Canada
- Quebec Centre for Biodiversity Science (QCBS), Montreal, Quebec, Canada
| | - Jerome Theau
- Department of Applied Geomatics, Université de Sherbrooke, Sherbrooke, Quebec, Canada
- Quebec Centre for Biodiversity Science (QCBS), Montreal, Quebec, Canada
| | - Samuel Foucher
- Department of Applied Geomatics, Université de Sherbrooke, Sherbrooke, Quebec, Canada
| |
Collapse
|
5
|
Kerry RG, Montalbo FJP, Das R, Patra S, Mahapatra GP, Maurya GK, Nayak V, Jena AB, Ukhurebor KE, Jena RC, Gouda S, Majhi S, Rout JR. An overview of remote monitoring methods in biodiversity conservation. ENVIRONMENTAL SCIENCE AND POLLUTION RESEARCH INTERNATIONAL 2022; 29:80179-80221. [PMID: 36197618 PMCID: PMC9534007 DOI: 10.1007/s11356-022-23242-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Accepted: 09/20/2022] [Indexed: 06/16/2023]
Abstract
Conservation of biodiversity is critical for the coexistence of humans and the sustenance of other living organisms within the ecosystem. Identification and prioritization of specific regions to be conserved are impossible without proper information about the sites. Advanced monitoring agencies like the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) had accredited that the sum total of species that are now threatened with extinction is higher than ever before in the past and are progressing toward extinct at an alarming rate. Besides this, the conceptualized global responses to these crises are still inadequate and entail drastic changes. Therefore, more sophisticated monitoring and conservation techniques are required which can simultaneously cover a larger surface area within a stipulated time frame and gather a large pool of data. Hence, this study is an overview of remote monitoring methods in biodiversity conservation via a survey of evidence-based reviews and related studies, wherein the description of the application of some technology for biodiversity conservation and monitoring is highlighted. Finally, the paper also describes various transformative smart technologies like artificial intelligence (AI) and/or machine learning algorithms for enhanced working efficiency of currently available techniques that will aid remote monitoring methods in biodiversity conservation.
Collapse
Affiliation(s)
- Rout George Kerry
- Department of Biotechnology, Utkal University, Vani Vihar, Bhubaneswar, Odisha 751004 India
| | | | - Rajeswari Das
- Department of Soil Science and Agricultural Chemistry, School of Agriculture, GIET University, Gunupur, Rayagada, Odisha 765022 India
| | - Sushmita Patra
- Indian Council of Agricultural Research-Directorate of Foot and Mouth Disease-International Centre for Foot and Mouth Disease, Arugul, Bhubaneswar, Odisha 752050 India
| | | | - Ganesh Kumar Maurya
- Zoology Section, Mahila MahaVidyalya, Banaras Hindu University, Varanasi, 221005 India
| | - Vinayak Nayak
- Indian Council of Agricultural Research-Directorate of Foot and Mouth Disease-International Centre for Foot and Mouth Disease, Arugul, Bhubaneswar, Odisha 752050 India
| | - Atala Bihari Jena
- Department of Neurosurgery, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02115 USA
| | | | - Ram Chandra Jena
- Department of Pharmaceutical Sciences, Utkal University, Vani Vihar, Bhubaneswar, Odisha 751004 India
| | - Sushanto Gouda
- Department of Zoology, Mizoram University, Aizawl, 796009 India
| | - Sanatan Majhi
- Department of Biotechnology, Utkal University, Vani Vihar, Bhubaneswar, Odisha 751004 India
| | - Jyoti Ranjan Rout
- School of Biological Sciences, AIPH University, Bhubaneswar, Odisha 752101 India
| |
Collapse
|
6
|
LaRue M, Brooks C, Wege M, Salas L, Gardiner N. High‐resolution satellite imagery meets the challenge of monitoring remote marine protected areas in the Antarctic and beyond. Conserv Lett 2022. [DOI: 10.1111/conl.12884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Affiliation(s)
- Michelle LaRue
- Gateway Antarctica, School of Earth and Environment University of Canterbury Christchurch New Zealand
- Department of Earth and Environmental Sciences University of Minnesota Minneapolis Minnesota USA
| | - Cassandra Brooks
- Department of Environmental Studies University of Colorado‐Boulder Boulder Colorado USA
| | - Mia Wege
- Gateway Antarctica, School of Earth and Environment University of Canterbury Christchurch New Zealand
- Department of Zoology and Entomology University of Pretoria Pretoria South Africa
| | | | - Natasha Gardiner
- Gateway Antarctica, School of Earth and Environment University of Canterbury Christchurch New Zealand
- Antarctica New Zealand Christchurch New Zealand
| |
Collapse
|
7
|
Semi-Supervised Segmentation for Coastal Monitoring Seagrass Using RPA Imagery. REMOTE SENSING 2021. [DOI: 10.3390/rs13091741] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Intertidal seagrass plays a vital role in estimating the overall health and dynamics of coastal environments due to its interaction with tidal changes. However, most seagrass habitats around the globe have been in steady decline due to human impacts, disturbing the already delicate balance in the environmental conditions that sustain seagrass. Miniaturization of multi-spectral sensors has facilitated very high resolution mapping of seagrass meadows, which significantly improves the potential for ecologists to monitor changes. In this study, two analytical approaches used for classifying intertidal seagrass habitats are compared—Object-based Image Analysis (OBIA) and Fully Convolutional Neural Networks (FCNNs). Both methods produce pixel-wise classifications in order to create segmented maps. FCNNs are an emerging set of algorithms within Deep Learning. Conversely, OBIA has been a prominent solution within this field, with many studies leveraging in-situ data and multiresolution segmentation to create habitat maps. This work demonstrates the utility of FCNNs in a semi-supervised setting to map seagrass and other coastal features from an optical drone survey conducted at Budle Bay, Northumberland, England. Semi-supervision is also an emerging field within Deep Learning that has practical benefits of achieving state of the art results using only subsets of labelled data. This is especially beneficial for remote sensing applications where in-situ data is an expensive commodity. For our results, we show that FCNNs have comparable performance with the standard OBIA method used by ecologists.
Collapse
|
8
|
The Potential of Satellite Imagery for Surveying Whales. SENSORS 2021; 21:s21030963. [PMID: 33535463 PMCID: PMC7867100 DOI: 10.3390/s21030963] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Revised: 01/26/2021] [Accepted: 01/28/2021] [Indexed: 11/16/2022]
Abstract
The emergence of very high-resolution (VHR) satellite imagery (less than 1 m spatial resolution) is creating new opportunities within the fields of ecology and conservation biology. The advancement of sub-meter resolution imagery has provided greater confidence in the detection and identification of features on the ground, broadening the realm of possible research questions. To date, VHR imagery studies have largely focused on terrestrial environments; however, there has been incremental progress in the last two decades for using this technology to detect cetaceans. With advances in computational power and sensor resolution, the feasibility of broad-scale VHR ocean surveys using VHR satellite imagery with automated detection and classification processes has increased. Initial attempts at automated surveys are showing promising results, but further development is necessary to ensure reliability. Here we discuss the future directions in which VHR satellite imagery might be used to address urgent questions in whale conservation. We highlight the current challenges to automated detection and to extending the use of this technology to all oceans and various whale species. To achieve basin-scale marine surveys, currently not feasible with any traditional surveying methods (including boat-based and aerial surveys), future research requires a collaborative effort between biology, computation science, and engineering to overcome the present challenges to this platform’s use.
Collapse
|