Can volunteers collect data that are comparable to professional scientists? A study of variables used in monitoring the outcomes of ecosystem rehabilitation

Monitoring biological water quality by volunteers complements professional assessments

Progressively more community initiatives have been undertaken over last decades to monitor water quality. Biological data collected by volunteers has been used for biodiversity and water quality studies. Despite the many citizen science projects collecting and using macroinvertebrates, the number of scientific peer-reviewed publications that use this data, remains limited. In 2018, a citizen science project on biological water quality assessment was launched in the Netherlands. In this project, volunteers collect macroinvertebrates from a nearby waterbody, identify and count the number of specimens, and register the catch through a web portal to instantaneously receive a water quality score based on their data. Water quality monitoring in the Netherlands is traditionally the field of professionals working at water authorities. Here, we compare the data from the citizen science project with the data gathered by professionals. We evaluate information regarding type and distribution of sampled waterbodies and sampling period, and compare general patterns in both datasets with respect to collected animals and calculated water quality scores. The results show that volunteers and professionals seldomly sample the same waterbody, that there is some overlap in sampling period, and that volunteers more frequently sampled urban waters and smaller waterbodies. The citizen science project is thus yielding data about understudied waters and this spatial and temporal complementarity is useful. The character and thoroughness of the assessments by volunteers and professionals are likely to differentiate. Volunteers collected significantly lower numbers of animals per sample and fewer animals from soft sediments like worms and more mobile individuals from the open water column such as boatsmen and beetles. Due to the lack of simultaneous observations at various locations by volunteers and professionals, a direct comparison of water quality scores is impossible. However, the obtained patterns from both datasets show that the water quality scores between volunteers and professionals are dissimilar for the different water types. To bridge these differences, new tools and processes need to be further developed to increase the value of monitoring biological water quality by volunteers for professionals.

Alleviating Environmental Health Disparities Through Community Science and Data Integration

Environmental contamination is a fundamental determinant of health and well-being, and when the environment is compromised, vulnerabilities are generated. The complex challenges associated with environmental health and food security are influenced by current and emerging political, social, economic, and environmental contexts. To solve these “wicked” dilemmas, disparate public health surveillance efforts are conducted by local, state, and federal agencies. More recently, citizen/community science (CS) monitoring efforts are providing site-specific data. One of the biggest challenges in using these government datasets, let alone incorporating CS data, for a holistic assessment of environmental exposure is data management and interoperability. To facilitate a more holistic perspective and approach to solution generation, we have developed a method to provide a common data model that will allow environmental health researchers working at different scales and research domains to exchange data and ask new questions. We anticipate that this method will help to address environmental health disparities, which are unjust and avoidable, while ensuring CS datasets are ethically integrated to achieve environmental justice. Specifically, we used a transdisciplinary research framework to develop a methodology to integrate CS data with existing governmental environmental monitoring and social attribute data (vulnerability and resilience variables) that span across 10 different federal and state agencies. A key challenge in integrating such different datasets is the lack of widely adopted ontologies for vulnerability and resiliency factors. In addition to following the best practice of submitting new term requests to existing ontologies to fill gaps, we have also created an application ontology, the Superfund Research Project Data Interface Ontology (SRPDIO).

Chimpanzee identification and social network construction through an online citizen science platform

Citizen science has grown rapidly in popularity in recent years due to its potential to educate and engage the public while providing a means to address a myriad of scientific questions. However, the rise in popularity of citizen science has also been accompanied by concerns about the quality of data emerging from citizen science research projects. We assessed data quality in the online citizen scientist platform Chimp&See, which hosts camera trap videos of chimpanzees (Pan troglodytes) and other species across Equatorial Africa. In particular, we compared detection and identification of individual chimpanzees by citizen scientists with that of experts with years of experience studying those chimpanzees. We found that citizen scientists typically detected the same number of individual chimpanzees as experts, but assigned far fewer identifications (IDs) to those individuals. Those IDs assigned, however, were nearly always in agreement with the IDs provided by experts. We applied the data sets of citizen scientists and experts by constructing social networks from each. We found that both social networks were relatively robust and shared a similar structure, as well as having positively correlated individual network positions. Our findings demonstrate that, although citizen scientists produced a smaller data set based on fewer confirmed IDs, the data strongly reflect expert classifications and can be used for meaningful assessments of group structure and dynamics. This approach expands opportunities for social research and conservation monitoring in great apes and many other individually identifiable species.

Avian monitoring – comparing structured and unstructured citizen science

Context Citizen science is increasingly used to collect biodiversity data to inform conservation management, but its validity within urban greenspaces remains largely unresolved. Aims To assess the validity of eBird data for generating biodiversity estimates within an urban greenspace. Methods We compared data from structured avian surveys with eBird data at an urban greenspace in Sydney during 2012–16, using species richness and Shannon diversity indices. We also compared community composition, using non-metric multidimensional scaling (NMDS) and dissimilarities using non-parametric MANOVA. Key results Structured surveys had a lower overall species richness (80 versus 116) and Shannon diversity (3.64 versus 3.94) than eBird data, but we found no significant differences when using years as replicates. After standardising the richness and diversity indices by time spent surveying in a given year, structured surveys produced significantly higher biodiversity estimates. Further, when grouped into species occupying different broad habitats, there were no significant differences in waterbird or landbird species richness, or in Shannon diversity between data sources. Conclusions The most likely explanation for the larger magnitudes of the biodiversity indices from the eBird data is the increase in effort manifested in the number of observers, time spent surveying and spatial coverage. This resulted in increased detection of uncommon species, which in turn accounted for a significant difference (R2 = 0.21, P = 0.015) in overall community composition measured by the two methods. Implications Our results highlight the opportunities provided by eBird data as a useful tool for land managers for monitoring avian communities in urban areas.

Quality of non-expert citizen science data collected for habitat type conservation status assessment in Natura 2000 protected areas

EU biodiversity conservation policy is based on the Habitats Directive (92/43/EC), which aims that habitat types and species of Community interest should reach ‘favourable conservation status’. To this end, Member States are obliged to perform periodic assessment of species and habitat conservation status through biodiversity monitoring, which, in almost all cases, was performed by experts implementing standardized field protocols. Here, we examine the quality of data collected in the field by non-experts (citizen scientists) for the conservation status assessment of habitat types, and specifically for the criteria ‘typical species’, ‘specific structures and functions’, and ‘pressures and threats’. This task is complicated and demands different types of field data. We visited two Natura 2000 sites and investigated four habitat types (two in each site) with non-experts and compared their data to the data collected by experts for accuracy, completeness and spatial arrangement. The majority of the non-expert data were accurate (i.e. non-experts recorded information they observed in the field), but they were incomplete (i.e. non-experts detected less information than the experts). Also, non-experts chose their sampling locations closer to the edge of the habitat, i.e. in more marginal conditions and thus in potentially more degraded conditions, than experts.

Citizen surveillance for environmental monitoring: combining the efforts of citizen science and crowdsourcing in a quantitative data framework

Citizen science and crowdsourcing have been emerging as methods to collect data for surveillance and/or monitoring activities. They could be gathered under the overarching term citizen surveillance. The discipline, however, still struggles to be widely accepted in the scientific community, mainly because these activities are not embedded in a quantitative framework. This results in an ongoing discussion on how to analyze and make useful inference from these data. When considering the data collection process, we illustrate how citizen surveillance can be classified according to the nature of the underlying observation process measured in two dimensions—the degree of observer reporting intention and the control in observer detection effort. By classifying the observation process in these dimensions we distinguish between crowdsourcing, unstructured citizen science and structured citizen science. This classification helps the determine data processing and statistical treatment of these data for making inference. Using our framework, it is apparent that published studies are overwhelmingly associated with structured citizen science, and there are well developed statistical methods for the resulting data. In contrast, methods for making useful inference from purely crowd-sourced data remain under development, with the challenges of accounting for the unknown observation process considerable. Our quantitative framework for citizen surveillance calls for an integration of citizen science and crowdsourcing and provides a way forward to solve the statistical challenges inherent to citizen-sourced data.

Influence of volunteer and project characteristics on data quality of biological surveys

Volunteer involvement in biological surveys is becoming common in conservation and ecology, prompting questions on the quality of data collected in such surveys. In a systematic review of the peer-reviewed literature on the quality of data collected by volunteers, we examined the characteristics of volunteers (e.g., age, prior knowledge) and projects (e.g., systematic vs. opportunistic monitoring schemes) that affect data quality with regards to standardization of sampling, accuracy and precision of data collection, spatial and temporal representation of data, and sample size. Most studies (70%, n = 71) focused on the act of data collection. The majority of assessments of volunteer characteristics (58%, n = 93) examined the effect of prior knowledge and experience on quality of the data collected, often by comparing volunteers with experts or professionals, who were usually assumed to collect higher quality data. However, when both groups' data were compared with the same accuracy standard, professional data were more accurate in only 4 of 7 cases. The few studies that measured precision of volunteer and professional data did not conclusively show that professional data were less variable than volunteer data. To improve data quality, studies recommended changes to survey protocols, volunteer training, statistical analyses, and project structure (e.g., volunteer recruitment and retention).

The invisible prevalence of citizen science in global research: migratory birds and climate change

Citizen science is a research practice that relies on public contributions of data. The strong recognition of its educational value combined with the need for novel methods to handle subsequent large and complex data sets raises the question: Is citizen science effective at science? A quantitative assessment of the contributions of citizen science for its core purpose--scientific research--is lacking. We examined the contribution of citizen science to a review paper by ornithologists in which they formulated ten central claims about the impact of climate change on avian migration. Citizen science was never explicitly mentioned in the review article. For each of the claims, these ornithologists scored their opinions about the amount of research effort invested in each claim and how strongly the claim was supported by evidence. This allowed us to also determine whether their trust in claims was, unwittingly or not, related to the degree to which the claims relied primarily on data generated by citizen scientists. We found that papers based on citizen science constituted between 24 and 77% of the references backing each claim, with no evidence of a mistrust of claims that relied heavily on citizen-science data. We reveal that many of these papers may not easily be recognized as drawing upon volunteer contributions, as the search terms "citizen science" and "volunteer" would have overlooked the majority of the studies that back the ten claims about birds and climate change. Our results suggest that the significance of citizen science to global research, an endeavor that is reliant on long-term information at large spatial scales, might be far greater than is readily perceived. To better understand and track the contributions of citizen science in the future, we urge researchers to use the keyword "citizen science" in papers that draw on efforts of non-professionals.

The invisible prevalence of citizen science in global research: migratory birds and climate change

Citizen science is a research practice that relies on public contributions of data. The strong recognition of its educational value combined with the need for novel methods to handle subsequent large and complex data sets raises the question: Is citizen science effective at science? A quantitative assessment of the contributions of citizen science for its core purpose – scientific research – is lacking. We examined the contribution of citizen science to a review paper by ornithologists in which they formulated ten central claims about the impact of climate change on avian migration. Citizen science was never explicitly mentioned in the review article. For each of the claims, these ornithologists scored their opinions about the amount of research effort invested in each claim and how strongly the claim was supported by evidence. This allowed us to also determine whether their trust in claims was, unwittingly or not, related to the degree to which the claims relied primarily on data generated by citizen scientists. We found that papers based on citizen science constituted between 24 and 77% of the references backing each claim, with no evidence of a mistrust of claims that relied heavily on citizen-science data. We reveal that many of these papers may not easily be recognized as drawing upon volunteer contributions, as the search terms “citizen science” and “volunteer” would have overlooked the majority of the studies that back the ten claims about birds and climate change. Our results suggest that the significance of citizen science to global research, an endeavor that is reliant on long-term information at large spatial scales, might be far greater than is readily perceived. To better understand and track the contributions of citizen science in the future, we urge researchers to use the keyword “citizen science” in papers that draw on efforts of non-professionals.