1
|
Yang F, Hu C, Liang A, Wang S, Su Y, Xu F. CECS-CLIP: Fusing Domain Knowledge for Rare Wildlife Detection Model. Animals (Basel) 2024; 14:2909. [PMID: 39409858 PMCID: PMC11476111 DOI: 10.3390/ani14192909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2024] [Revised: 09/20/2024] [Accepted: 10/03/2024] [Indexed: 10/20/2024] Open
Abstract
Accurate and efficient wildlife monitoring is essential for conservation efforts. Traditional image-based methods often struggle to detect small, occluded, or camouflaged animals due to the challenges posed by complex natural environments. To overcome these limitations, an innovative multimodal target detection framework is proposed in this study, which integrates textual information from an animal knowledge base as supplementary features to enhance detection performance. First, a concept enhancement module was developed, employing a cross-attention mechanism to fuse features based on the correlation between textual and image features, thereby obtaining enhanced image features. Secondly, a feature normalization module was developed, amplifying cosine similarity and introducing learnable parameters to continuously weight and transform image features, further enhancing their expressive power in the feature space. Rigorous experimental validation on a specialized dataset provided by the research team at Northwest A&F University demonstrates that our multimodal model achieved a 0.3% improvement in precision over single-modal methods. Compared to existing multimodal target detection algorithms, this model achieved at least a 25% improvement in AP and excelled in detecting small targets of certain species, significantly surpassing existing multimodal target detection model benchmarks. This study offers a multimodal target detection model integrating textual and image information for the conservation of rare and endangered wildlife, providing strong evidence and new perspectives for research in this field.
Collapse
Affiliation(s)
- Feng Yang
- School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China; (F.Y.); (C.H.); (A.L.); (S.W.); (Y.S.)
- Engineering Research Center for Forestry-Oriented Intelligent Information Processing, National Forestry and Grassland Administration, Beijing 100083, China
- State Key Laboratory of Efficient Production of Forest Resources, Beijing 100083, China
| | - Chunying Hu
- School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China; (F.Y.); (C.H.); (A.L.); (S.W.); (Y.S.)
| | - Aokang Liang
- School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China; (F.Y.); (C.H.); (A.L.); (S.W.); (Y.S.)
| | - Sheng Wang
- School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China; (F.Y.); (C.H.); (A.L.); (S.W.); (Y.S.)
| | - Yun Su
- School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China; (F.Y.); (C.H.); (A.L.); (S.W.); (Y.S.)
| | - Fu Xu
- School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China; (F.Y.); (C.H.); (A.L.); (S.W.); (Y.S.)
- Engineering Research Center for Forestry-Oriented Intelligent Information Processing, National Forestry and Grassland Administration, Beijing 100083, China
- State Key Laboratory of Efficient Production of Forest Resources, Beijing 100083, China
| |
Collapse
|
2
|
Mesaglio T, Sauquet H, Coleman D, Wenk E, Cornwell WK. Photographs as an essential biodiversity resource: drivers of gaps in the vascular plant photographic record. THE NEW PHYTOLOGIST 2023; 238:1685-1694. [PMID: 36913725 DOI: 10.1111/nph.18813] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/06/2022] [Accepted: 02/07/2023] [Indexed: 06/18/2023]
Abstract
The photographic record is increasingly becoming an important biodiversity resource for primary research and conservation monitoring. However, globally, there are important gaps in this record even in relatively well-researched floras. To quantify the gaps in the Australian native vascular plant photographic record, we systematically surveyed 33 sources of well-curated species photographs, assembling a list of species with accessible and verifiable photographs, as well as the species for which this search failed. Of 21 077 Australian native species, 3715 lack a verifiable photograph across our 33 surveyed resources. There are three major geographic hotspots of unphotographed species in Australia, all far from current population centres. Many unphotographed species are small in stature or uncharismatic, and many are also recently described. The large number of recently described species without accessible photographs was surprising. There are longstanding efforts in Australia to organise the plant photographic record, but in the absence of a global consensus to treat photographs as an essential biodiversity resource, this has not become common practice. Many recently described species are small-range endemics and some have special conservation status. Completing the botanical photographic record across the globe will facilitate a virtuous feedback loop of more efficient identification, monitoring and conservation.
Collapse
Affiliation(s)
- Thomas Mesaglio
- Evolution & Ecology Research Centre, School of Biological, Earth and Environmental Sciences, UNSW Sydney, Sydney, NSW, 2052, Australia
- Centre for Ecosystem Science, School of Biological, Earth and Environmental Sciences, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Hervé Sauquet
- Evolution & Ecology Research Centre, School of Biological, Earth and Environmental Sciences, UNSW Sydney, Sydney, NSW, 2052, Australia
- National Herbarium of New South Wales, Royal Botanic Gardens and Domain Trust, Sydney, NSW, 2000, Australia
| | - David Coleman
- Evolution & Ecology Research Centre, School of Biological, Earth and Environmental Sciences, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Elizabeth Wenk
- Evolution & Ecology Research Centre, School of Biological, Earth and Environmental Sciences, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - William K Cornwell
- Evolution & Ecology Research Centre, School of Biological, Earth and Environmental Sciences, UNSW Sydney, Sydney, NSW, 2052, Australia
| |
Collapse
|
3
|
Nazir S, Khan HU, Shahzad S, García-Magariño I. Editorial on decision support system for development of intelligent applications. Soft comput 2022. [DOI: 10.1007/s00500-022-07390-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|