1
|
Vizcarra JC, Pearce TM, Dugger BN, Keiser MJ, Gearing M, Crary JF, Kiely EJ, Morris M, White B, Glass JD, Farrell K, Gutman DA. Toward a generalizable machine learning workflow for neurodegenerative disease staging with focus on neurofibrillary tangles. Acta Neuropathol Commun 2023; 11:202. [PMID: 38110981 PMCID: PMC10726581 DOI: 10.1186/s40478-023-01691-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 11/19/2023] [Indexed: 12/20/2023] Open
Abstract
Machine learning (ML) has increasingly been used to assist and expand current practices in neuropathology. However, generating large imaging datasets with quality labels is challenging in fields which demand high levels of expertise. Further complicating matters is the often seen disagreement between experts in neuropathology-related tasks, both at the case level and at a more granular level. Neurofibrillary tangles (NFTs) are a hallmark pathological feature of Alzheimer disease, and are associated with disease progression which warrants further investigation and granular quantification at a scale not currently accessible in routine human assessment. In this work, we first provide a baseline of annotator/rater agreement for the tasks of Braak NFT staging between experts and NFT detection using both experts and novices in neuropathology. We use a whole-slide-image (WSI) cohort of neuropathology cases from Emory University Hospital immunohistochemically stained for Tau. We develop a workflow for gathering annotations of the early stage formation of NFTs (Pre-NFTs) and mature intracellular (iNFTs) and show ML models can be trained to learn annotator nuances for the task of NFT detection in WSIs. We utilize a model-assisted-labeling approach and demonstrate ML models can be used to aid in labeling large datasets efficiently. We also show these models can be used to extract case-level features, which predict Braak NFT stages comparable to expert human raters, and do so at scale. This study provides a generalizable workflow for various pathology and related fields, and also provides a technique for accomplishing a high-level neuropathology task with limited human annotations.
Collapse
Affiliation(s)
- Juan C Vizcarra
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, 313 Ferst Dr NW, Atlanta, GA, 30332, USA
| | - Thomas M Pearce
- Department of Pathology, Division of Neuropathology, University of Pittsburgh Medical Center, Room S701 Scaife Hall 3550 Terrace Street, Pittsburgh, PA, 15261, USA
| | - Brittany N Dugger
- Department of Pathology and Laboratory Medicine, University of California-Davis School of Medicine, 3400A Research Building III Sacramento, Davis, CA, 95817, USA
| | - Michael J Keiser
- Department of Pharmaceutical Chemistry, Department of Bioengineering and Therapeutic Sciences, Institute for Neurodegenerative Diseases, Kavli Institute for Fundamental Neuroscience, and Bakar Computational Health Sciences Institute, University of California, 675 Nelson Rising Ln, Box 0518, San Francisco, CA, 94143, USA
| | - Marla Gearing
- Department of Neurology, Emory University School of Medicine, 12 Executive Park Dr NE, Atlanta, GA, 30322, USA
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, 1364 Clifton Rd, Atlanta, GA, 30322, USA
| | - John F Crary
- Departments of Pathology, Neuroscience, and Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Neuropathology Brain Bank and Research Core, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Ronald M. Loeb Center for Alzheimer's Disease, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Pathology, Icahn School of Medicine at Mount Sinai, Icahn Building 9th Floor, Room 20A, 1425 Madison Avenue, New York, NY, 10029, USA
| | - Evan J Kiely
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, 1364 Clifton Rd, Atlanta, GA, 30322, USA
| | - Meaghan Morris
- Department of Pathology, Johns Hopkins School of Medicine, Baltimore, MD, 21218, USA
| | - Bartholomew White
- Department of Pathology, Beth Israel Deaconess Medical Center, 330 Brookline Avenue, Boston, MA, 02215, USA
| | - Jonathan D Glass
- Department of Neurology, Emory University School of Medicine, 12 Executive Park Dr NE, Atlanta, GA, 30322, USA
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, 1364 Clifton Rd, Atlanta, GA, 30322, USA
- Center for Neurodegenerative Disease, Emory University School of Medicine, Whitehead Biomedical Research Building, 615 Michael Street, 5th Floor, Suite 500, Atlanta, GA, 30322, USA
| | - Kurt Farrell
- Departments of Pathology, Neuroscience, and Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Neuropathology Brain Bank and Research Core, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Ronald M. Loeb Center for Alzheimer's Disease, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Department of Pathology, Icahn School of Medicine at Mount Sinai, Icahn Building 9th Floor, L9-02C, 1425 Madison, Avenue, New York, NY, USA
| | - David A Gutman
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, 1364 Clifton Rd, Atlanta, GA, 30322, USA.
| |
Collapse
|
2
|
Nieto-Mora D, Rodríguez-Buritica S, Rodríguez-Marín P, Martínez-Vargaz J, Isaza-Narváez C. Systematic review of machine learning methods applied to ecoacoustics and soundscape monitoring. Heliyon 2023; 9:e20275. [PMID: 37790981 PMCID: PMC10542774 DOI: 10.1016/j.heliyon.2023.e20275] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2023] [Revised: 09/12/2023] [Accepted: 09/18/2023] [Indexed: 10/05/2023] Open
Abstract
Soundscape ecology is a promising area that studies landscape patterns based on their acoustic composition. It focuses on the distribution of biotic and abiotic sounds at different frequencies of the landscape acoustic attribute and the relationship of said sounds with ecosystem health metrics and indicators (e.g., species richness, acoustic biodiversity, vectors of structural change, gradients of vegetation cover, landscape connectivity, and temporal and spatial characteristics). To conduct such studies, researchers analyze recordings from Acoustic Recording Units (ARUs). The increasing use of ARUs and their capacity to record hours of audio for months at a time have created a need for automatic processing methods to reduce time consumption, correlate variables implicit in the recordings, extract features, and characterize sound patterns related to landscape attributes. Consequently, traditional machine learning methods have been commonly used to process data on different characteristics of soundscapes, mainly the presence-absence of species. In addition, it has been employed for call segmentation, species identification, and sound source clustering. However, some authors highlight the importance of the new approaches that use unsupervised deep learning methods to improve the results and diversify the assessed attributes. In this paper, we present a systematic review of machine learning methods used in the field of ecoacoustics for data processing. It includes recent trends, such as semi-supervised and unsupervised deep learning methods. Moreover, it maintains the format found in the reviewed papers. First, we describe the ARUs employed in the papers analyzed, their configuration, and the study sites where the datasets were collected. Then, we provide an ecological justification that relates acoustic monitoring to landscape features. Subsequently, we explain the machine learning methods followed to assess various landscape attributes. The results show a trend towards label-free methods that can process the large volumes of data gathered in recent years. Finally, we discuss the need to adopt methods with a machine learning approach in other biological dimensions of landscapes.
Collapse
Affiliation(s)
- D.A. Nieto-Mora
- MIRP-Instituto Tecnológico Metropolitano ITM, Cl. 54a N∘30-01, Medellín, Colombia
| | | | | | | | | |
Collapse
|
3
|
Best P, Paris S, Glotin H, Marxer R. Deep audio embeddings for vocalisation clustering. PLoS One 2023; 18:e0283396. [PMID: 37428759 DOI: 10.1371/journal.pone.0283396] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Accepted: 06/25/2023] [Indexed: 07/12/2023] Open
Abstract
The study of non-human animals' communication systems generally relies on the transcription of vocal sequences using a finite set of discrete units. This set is referred to as a vocal repertoire, which is specific to a species or a sub-group of a species. When conducted by human experts, the formal description of vocal repertoires can be laborious and/or biased. This motivates computerised assistance for this procedure, for which machine learning algorithms represent a good opportunity. Unsupervised clustering algorithms are suited for grouping close points together, provided a relevant representation. This paper therefore studies a new method for encoding vocalisations, allowing for automatic clustering to alleviate vocal repertoire characterisation. Borrowing from deep representation learning, we use a convolutional auto-encoder network to learn an abstract representation of vocalisations. We report on the quality of the learnt representation, as well as of state of the art methods, by quantifying their agreement with expert labelled vocalisation types from 8 datasets of other studies across 6 species (birds and marine mammals). With this benchmark, we demonstrate that using auto-encoders improves the relevance of vocalisation representation which serves repertoire characterisation using a very limited number of settings. We also publish a Python package for the bioacoustic community to train their own vocalisation auto-encoders or use a pretrained encoder to browse vocal repertoires and ease unit wise annotation.
Collapse
Affiliation(s)
- Paul Best
- Université de Toulon, Aix Marseille Univ, CNRS, LIS, Toulon, France
| | - Sébastien Paris
- Université de Toulon, Aix Marseille Univ, CNRS, LIS, Toulon, France
| | - Hervé Glotin
- Université de Toulon, Aix Marseille Univ, CNRS, LIS, Toulon, France
| | - Ricard Marxer
- Université de Toulon, Aix Marseille Univ, CNRS, LIS, Toulon, France
| |
Collapse
|
4
|
Hildebrand JA, Frasier KE, Helble TA, Roch MA. Performance metrics for marine mammal signal detection and classification. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 151:414. [PMID: 35105012 DOI: 10.1121/10.0009270] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 12/15/2021] [Indexed: 06/14/2023]
Abstract
Automatic algorithms for the detection and classification of sound are essential to the analysis of acoustic datasets with long duration. Metrics are needed to assess the performance characteristics of these algorithms. Four metrics for performance evaluation are discussed here: receiver-operating-characteristic (ROC) curves, detection-error-trade-off (DET) curves, precision-recall (PR) curves, and cost curves. These metrics were applied to the generalized power law detector for blue whale D calls [Helble, Ierley, D'Spain, Roch, and Hildebrand (2012). J. Acoust. Soc. Am. 131(4), 2682-2699] and the click-clustering neural-net algorithm for Cuvier's beaked whale echolocation click detection [Frasier, Roch, Soldevilla, Wiggins, Garrison, and Hildebrand (2017). PLoS Comp. Biol. 13(12), e1005823] using data prepared for the 2015 Detection, Classification, Localization and Density Estimation Workshop. Detection class imbalance, particularly the situation of rare occurrence, is common for long-term passive acoustic monitoring datasets and is a factor in the performance of ROC and DET curves with regard to the impact of false positive detections. PR curves overcome this shortcoming when calculated for individual detections and do not rely on the reporting of true negatives. Cost curves provide additional insight on the effective operating range for the detector based on the a priori probability of occurrence. Use of more than a single metric is helpful in understanding the performance of a detection algorithm.
Collapse
Affiliation(s)
- John A Hildebrand
- Scripps Institution of Oceanography, University of California San Diego, La Jolla, California 92093, USA
| | - Kaitlin E Frasier
- Scripps Institution of Oceanography, University of California San Diego, La Jolla, California 92093, USA
| | - Tyler A Helble
- Naval Information Warfare Center Pacific, San Diego, California 92152, USA
| | - Marie A Roch
- Department of Computer Science, San Diego State University, San Diego, California 92182, USA
| |
Collapse
|