Leveling up citizen science

Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification

OBJECTIVES

Segmentation is crucial in medical imaging. Deep learning based on convolutional neural networks showed promising results. However, the absence of large-scale datasets and a high degree of inter- and intra-observer variations pose a bottleneck. Crowdsourcing might be an alternative, as many non-experts provide references. We aim to compare different types of crowdsourcing for medical image segmentation.

METHODS

We develop a crowdsourcing platform that integrates citizen science (incentive: participating in the research), paid microtask (incentive: financial reward), and gamification (incentive: entertainment). For evaluation, we choose the use case of sclera segmentation in fundus images as a proof-of-concept and analyze the accuracy of crowdsourced masks and the generalization of learning models trained with crowdsourced masks.

RESULTS

The developed platform is suited for the different types of crowdsourcing and offers an easy and intuitive way to implement crowdsourcing studies. Regarding the proof-of-concept study, citizen science, paid microtask, and gamification yield a median F-score of 82.2, 69.4, and 69.3 % compared to expert-labeled ground truth, respectively. Generating consensus masks improves the gamification masks (78.3 %). Despite the small training data (50 images), deep learning reaches median F-scores of 80.0, 73.5, and 76.5 % for citizen science, paid microtask, and gamification, respectively, indicating sufficient generalizability.

CONCLUSIONS

As the platform has proven useful, we aim to make it available as open-source software for other researchers.

Social Justice, Community Engagement, and Undergraduate STEM Education: Participatory Science as a Teaching Tool

Social justice is increasingly being seen as relevant to the science curriculum. We examine the intersection of participatory science, social justice, and higher education in the United States to investigate how instructors can teach about social justice and enhance collaborations to work toward enacting social justice. Participatory science approaches, like those that collect data over large geographic areas, can be particularly useful for teaching students about social justice. Conversely, local-scale approaches that integrate students into community efforts can create powerful collaborations to help facilitate social justice. We suggest a variety of large-scale databases, platforms, and portals that could be used as starting points to address a set of learning objectives about social justice. We also describe local-scale participatory science approaches with a social justice focus, developed through academic and community partnerships. Considerations for implementing participatory science with undergraduates are discussed, including cautions about the necessary time investment, cultural competence, and institutional support. These approaches are not always appropriate but can provide compelling learning experiences in the correct circumstances.

Improving microbial phylogeny with citizen science within a mass-market video game

Citizen science video games are designed primarily for users already inclined to contribute to science, which severely limits their accessibility for an estimated community of 3 billion gamers worldwide. We created Borderlands Science (BLS), a citizen science activity that is seamlessly integrated within a popular commercial video game played by tens of millions of gamers. This integration is facilitated by a novel game-first design of citizen science games, in which the game design aspect has the highest priority, and a suitable task is then mapped to the game design. BLS crowdsources a multiple alignment task of 1 million 16S ribosomal RNA sequences obtained from human microbiome studies. Since its initial release on 7 April 2020, over 4 million players have solved more than 135 million science puzzles, a task unsolvable by a single individual. Leveraging these results, we show that our multiple sequence alignment simultaneously improves microbial phylogeny estimations and UniFrac effect sizes compared to state-of-the-art computational methods. This achievement demonstrates that hyper-gamified scientific tasks attract massive crowds of contributors and offers invaluable resources to the scientific community.

When online citizen science meets teaching: Storyfication of a science discovery game to teach, learn, and contribute to genomic research

In the last decade, video games became a common vehicle for citizen science initiatives in life science, allowing participants to contribute to real scientific data analysis while learning about it. Since 2010, our scientific discovery game (SDG) Phylo enlists participants in comparative genomic data analysis. It is frequently used as a learning tool, but the activities were difficult to aggregate to build a coherent teaching activity. Here, we describe a strategy and series of recipes to facilitate the integration of SDGs in courses and implement this approach in Phylo. We developed new roles and functionalities enabling instructors to create assignments and monitor the progress of students. A story mode progressively introduces comparative genomics concepts, allowing users to learn and contribute to the analysis of real genomic sequences. Preliminary results from a user study suggest this framework may help to boost user motivation and clarify pedagogical objectives.

Quantifying online citizen science: Dynamics and demographics of public participation in science

Citizen scientists around the world are collecting data with their smartphones, performing scientific calculations on their home computers, and analyzing images on online platforms. These online citizen science projects are frequently lauded for their potential to revolutionize the scope and scale of data collection and analysis, improve scientific literacy, and democratize science. Yet, despite the attention online citizen science has attracted, it remains unclear how widespread public participation is, how it has changed over time, and how it is geographically distributed. Importantly, the demographic profile of citizen science participants remains uncertain, and thus to what extent their contributions are helping to democratize science. Here, we present the largest quantitative study of participation in citizen science based on online accounts of more than 14 million participants over two decades. We find that the trend of broad rapid growth in online citizen science participation observed in the early 2000s has since diverged by mode of participation, with consistent growth observed in nature sensing, but a decline seen in crowdsourcing and distributed computing. Most citizen science projects, except for nature sensing, are heavily dominated by men, and the vast majority of participants, male and female, have a background in science. The analysis we present here provides, for the first time, a robust 'baseline' to describe global trends in online citizen science participation. These results highlight current challenges and the future potential of citizen science. Beyond presenting our analysis of the collated data, our work identifies multiple metrics for robust examination of public participation in science and, more generally, online crowds. It also points to the limits of quantitative studies in capturing the personal, societal, and historical significance of citizen science.

New Method for Rapid 3D Reconstruction of Semi-Transparent Underwater Animals and Structures

Morphological features are the primary identifying properties of most animals and key to many comparative physiological studies, yet current techniques for preservation and documentation of soft-bodied marine animals are limited in terms of quality and accessibility. Digital records can complement physical specimens, with a wide array of applications ranging from species description to kinematics modeling, but options are lacking for creating models of soft-bodied semi-transparent underwater animals. We developed a lab-based technique that can live-scan semi-transparent, submerged animals, and objects within seconds. To demonstrate the method, we generated full three-dimensional reconstructions (3DRs) of an object of known dimensions for verification, as well as two live marine animals-a siphonophore and an amphipod-allowing detailed measurements on each. Techniques like these pave the way for faster data capture, integrative and comparative quantitative approaches, and more accessible collections of fragile and rare biological samples.

Practical recommendations from a multi-perspective needs and challenges assessment of citizen science games

Citizen science games are an increasingly popular form of citizen science, in which volunteer participants engage in scientific research while playing a game. Their success depends on a diverse set of stakeholders working together-scientists, volunteers, and game developers. Yet the potential needs of these stakeholder groups and their possible tensions are poorly understood. To identify these needs and possible tensions, we conducted a qualitative data analysis of two years of ethnographic research and 57 interviews with stakeholders from 10 citizen science games, following a combination of grounded theory and reflexive thematic analysis. We identify individual stakeholder needs as well as important barriers to citizen science game success. These include the ambiguous allocation of developer roles, limited resources and funding dependencies, the need for a citizen science game community, and science-game tensions. We derive recommendations for addressing these barriers.

FathomNet: A global image database for enabling artificial intelligence in the ocean

The ocean is experiencing unprecedented rapid change, and visually monitoring marine biota at the spatiotemporal scales needed for responsible stewardship is a formidable task. As baselines are sought by the research community, the volume and rate of this required data collection rapidly outpaces our abilities to process and analyze them. Recent advances in machine learning enables fast, sophisticated analysis of visual data, but have had limited success in the ocean due to lack of data standardization, insufficient formatting, and demand for large, labeled datasets. To address this need, we built FathomNet, an open-source image database that standardizes and aggregates expertly curated labeled data. FathomNet has been seeded with existing iconic and non-iconic imagery of marine animals, underwater equipment, debris, and other concepts, and allows for future contributions from distributed data sources. We demonstrate how FathomNet data can be used to train and deploy models on other institutional video to reduce annotation effort, and enable automated tracking of underwater concepts when integrated with robotic vehicles. As FathomNet continues to grow and incorporate more labeled data from the community, we can accelerate the processing of visual data to achieve a healthy and sustainable global ocean.

Public Engagement with Biotechnology Inside and Outside the Classroom: Community-Focused Approaches

Biotechnology offers vast benefits to the environment, animals, and human health, and contributes to improving socioeconomic conditions for the public. However, biotechnology innovations continue to trigger public concern and opposition over their potential social, health, and ecological risks. There is an opportunity to increase knowledge and acceptance of biotechnology through engagement, education, and community participation. In this perspective, we highlight crucial factors that shape the public perception of biotechnology and present opportunities for scientists to effectively communicate their ideas while engaging with local and global communities. Initiatives that seek to involve communities in design, development, and adoption processes are crucial for the successful implementation of biotechnology-based solutions.

Human-supervised clustering of multidimensional data using crowdsourcing

Clustering is a central task in many data analysis applications. However, there is no universally accepted metric to decide the occurrence of clusters. Ultimately, we have to resort to a consensus between experts. The problem is amplified with high-dimensional datasets where classical distances become uninformative and the ability of humans to fully apprehend the distribution of the data is challenged. In this paper, we design a mobile human-computing game as a tool to query human perception for the multidimensional data clustering problem. We propose two clustering algorithms that partially or entirely rely on aggregated human answers and report the results of two experiments conducted on synthetic and real-world datasets. We show that our methods perform on par or better than the most popular automated clustering algorithms. Our results suggest that hybrid systems leveraging annotations of partial datasets collected through crowdsourcing platforms can be an efficient strategy to capture the collective wisdom for solving abstract computational problems.

Inclusion in citizen science: The conundrum of rebranding

Does replacing the term “citizen science” do more harm than good?