Understanding crowdsourcing projects: A systematic review of tendencies, workflow, and quality management

Leveraging collective action and environmental literacy to address complex sustainability challenges

Developing and enhancing societal capacity to understand, debate elements of, and take actionable steps toward a sustainable future at a scale beyond the individual are critical when addressing sustainability challenges such as climate change, resource scarcity, biodiversity loss, and zoonotic disease. Although mounting evidence exists for how to facilitate individual action to address sustainability challenges, there is less understanding of how to foster collective action in this realm. To support research and practice promoting collective action to address sustainability issues, we define the term "collective environmental literacy" by delineating four key potent aspects: scale, dynamic processes, shared resources, and synergy. Building on existing collective constructs and thought, we highlight areas where researchers, practitioners, and policymakers can support individuals and communities as they come together to identify, develop, and implement solutions to wicked problems. We close by discussing limitations of this work and future directions in studying collective environmental literacy.

Towards a typology development of crowdsourcing in science

Crowdsourcing in science as collaborative online process through which non-professional and/or professional scientists incorporate a group of individuals of varying, diversity knowledge and skills, via an open call to the Internet and/or online platforms, to undertaking of a task in science, is an important strategy to support scientific research that has gained attention in academia and practitioners. While research efforts to date have focused on the benefits of crowdsourcing in science, its typology has yet to mature. Typologies are important in describing complex, multidisciplinary organisational forms such as crowdsourcing in science. The main purpose of this article is to identify and provide a typology of crowdsourcing in science. Based on the thematic analysis of publications collected in a systematic manner and focused group interviews, 12 types of crowdsourcing in science are identified. The proposed crowdsourcing in science typology matrix may be a starting point for future research and decision-making by practitioners regarding the choice of a specific type of crowdsourcing in science.

KAZNET: An Open-Source, Micro-Tasking Platform for Remote Locations

Field surveys are the workhorse of social and environmental research, but conventional collection through monitors or enumerators are cost prohibitive in many remote or otherwise difficult settings, which can lead to a poor understanding of those environments and an underrepresentation of the people living in them. In such cases, micro-tasking can offer a promising alternative. By activating in-situ data collectors, micro-tasking avoids many of the large expenses related to conventional field survey processes. In addition to relaxing resource constraints, crowd-sourcing can be flexible and employ data quality protocols unheard-of for conventional methods. This study assesses the potential of using micro-tasking to monitor socioeconomic and environmental indicators in remote settings using a new platform called KAZNET. KAZNET leverages the network of people with smartphones, which are becoming ubiquitous even in the remote rural settings, to execute both long-term and short-term data collection activities, with flexibility to adjust or add tasks in real-time. It also allows for multiple projects, requiring different data types, to be rolled out in the same platform simultaneously. For the data-collector, KAZNET is effectively a wrapper for the commonly used and open source, Open Data Kit (ODK) software, which specializes in offline data collection. A web interface allows administrators to calibrate, deploy, and validate tasks performed by contributors. KAZNET has been used in several projects to collect data in remote pastoral regions of East Africa since its inception in 2017. KAZNET has shown to be effective for collecting high frequency and repeated measures from markets, households and rangelands in remote regions at relatively low cost compared to traditional survey methods. While the successes of micro-tasking are promising, there are clear trade-offs and complementarities between micro-tasking and standard surveys methods, which researchers and practitioners need to consider when implementing either approach.

A TOPSIS-Inspired Ranking Method Using Constrained Crowd Opinions for Urban Planning

Crowdsourcing has become an important tool for gathering knowledge for urban planning problems. The questions posted to the crowd for urban planning problems are quite different from the traditional crowdsourcing models. Unlike the traditional crowdsourcing models, due to the constraints among the multiple components (e.g., multiple locations of facilities) in a single question and non-availability of the defined option sets, aggregating of multiple diverse opinions that satisfy the constraints as well as finding the ranking of the crowd workers becomes challenging. Moreover, owing to the presence of the conflicting nature of features, the traditional ranking methods such as the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) cannot always be feasible as the optimal solutions in terms of multiple objectives cannot occur simultaneously for the conflicting cases (e.g., benefit and cost criteria) for urban planning problems. Therefore, in this work, a multi-objective approach is proposed to produce better compromised solutions in terms of conflicting features from the general crowd. In addition, the solutions are employed to obtain a proper ideal solution for ranking the crowd. The experimental results are validated using two constrained crowd opinion datasets for real-world urban planning problems and compared with the state-of-the-art TOPSIS models.

Data Fusion in Earth Observation and the Role of Citizen as a Sensor: A Scoping Review of Applications, Methods and Future Trends

Recent advances in Earth Observation (EO) placed Citizen Science (CS) in the highest position, declaring their essential provision of information in every discipline that serves the SDGs, and the 2050 climate neutrality targets. However, so far, none of the published literature reviews has investigated the models and tools that assimilate these data sources. Following this gap of knowledge, we synthesised this scoping systematic literature review (SSLR) with a will to cover this limitation and highlight the benefits and the future directions that remain uncovered. Adopting the SSLR guidelines, a double and two-level screening hybrid process found 66 articles to meet the eligibility criteria, presenting methods, where data were fused and evaluated regarding their performance, scalability level and computational efficiency. Subsequent reference is given on EO-data, their corresponding conversions, the citizens’ participation digital tools, and Data Fusion (DF) models that are predominately exploited. Preliminary results showcased a preference in the multispectral satellite sensors, with the microwave sensors to be used as a supplementary data source. Approaches such as the “brute-force approach” and the super-resolution models indicate an effective way to overcome the spatio-temporal gaps and the so far reliance on commercial satellite sensors. Passive crowdsensing observations are foreseen to gain a greater audience as, described in, most cases as a low-cost and easily applicable solution even in the unprecedented COVID-19 pandemic. Immersive platforms and decentralised systems should have a vital role in citizens’ engagement and training process. Reviewing the DF models, the majority of the selected articles followed a data-driven method with the traditional algorithms to still hold significant attention. An exception is revealed in the smaller-scale studies, which showed a preference for deep learning models. Several studies enhanced their methods with the active-, and transfer-learning approaches, constructing a scalable model. In the end, we strongly support that the interaction with citizens is of paramount importance to achieve a climate-neutral Earth.

Essential elements, conceptual foundations and workflow design in crowd-powered projects

Crowdsourcing arose as a problem-solving strategy that uses a large number of workers to achieve tasks and solve specific problems. Although there are many studies that explore crowdsourcing platforms and systems, little attention has been paid to define what a crowd-powered project is. To address this issue, this article introduces a general-purpose conceptual model that represents the essential elements involved in this kind of project and how they relate to each other. We consider that the workflow in crowdsourcing projects is context-oriented and should represent the planning and coordination by the crowdsourcer in the project, instead of only facilitating decomposing a complex task into subtask sets. Since structural models are limited to cannot properly represent the execution flow, we also introduce the use of behavioural conceptual models, specifically Unified Modeling Language (UML) activity diagrams, to represent the user, tasks, assets, control activities and products involved in a specific project.

Evaluation of pharmaceutical pictograms by older "turkers": A cross-sectional crowdsourced study

BACKGROUND

Well-designed pharmaceutical pictograms may improve patients' understanding of medication instructions. However, the iterative participatory design process required to produce effective pictograms can be costly in terms of money, time, and effort. Crowdsourcing has been applied to bring down the costs of the participatory design process, but the feasibility of using this approach with older adults remains largely unknown.

OBJECTIVES

To evaluate the feasibility of using Amazon Mechanical Turk (MTurk), a leading crowdsourcing platform, for participatory pictogram evaluation with older adults (55+) and to evaluate the comprehensibility of USP pictogram, identify common misinterpretations, and explore the relationship between selected participant characteristics and their pictogram comprehension performance.

METHODS

108 older adults (56.5% female; 57-80 years of age) were recruited via MTurk to complete a cross-sectional online survey that asked them to interpret 15 USP pictograms and answer questions about their health and health literacy.

RESULTS

It was feasible to perform pictogram evaluation with older adults on MTurk, as shown by ease of recruitment and high data quality. Of the 15 pictograms tested, seven (46.7%) resulted in a comprehensibility score below the threshold established by the American National Standards Institute (ANSI), eight (53.3%) elicited common misinterpretations, and two (13.3%) resulted in ANSI-defined "critical confusion." Age (P = 0.04) was associated with pictogram comprehension performance. Certain issues with the pictogram subtitles emerged during the evaluation.

CONCLUSIONS

MTurk is a feasible platform for participatory pictogram evaluation, even for a sole target of older adults. The USP should develop a pictogram user manual, redesign pictograms confusing to older adults, and establish policies and procedures to ensure that pictogram subtitles conform to evidence-based best practices and standards for patient-centered written drug information.

Self-evaluation of knowledge sharing through the lens of social comparison theory

Purpose

The purpose of this study is to investigate how different types of contribution awareness information influence knowledge sharing motivation and contribution persistence.

Design/methodology/approach

The independent variable of this experimental study was contribution awareness information with four levels: self-contribution, absolute social-comparison, relative social-comparison and control. The dependent variables were self-rated knowledge sharing motivation measured on a six-point Likert scale and contribution persistence measured by number of contributions. A total of 182 knowledge workers voluntarily completed online participation. Participants were randomly assigned to one of the four intervention groups.

Findings

The study found that the self-contribution group outperformed the other groups in both knowledge sharing motivation and contribution persistence; this observation was significant compared with the absolute social-comparison and control groups. The impact of self-contribution frequency information was stronger for contribution persistence than for self-evaluated knowledge sharing motivation, highlighting the gap between perception and behavior. It is also noteworthy that comparative information negatively influenced knowledge sharing motivation and contribution persistence, implying that social comparison played a role in priming individuals to focus on dissimilarities between the comparison target and themselves.

Originality/value

This study provides behavior-based evidence supporting social comparison theory and the selective accessibility model in the field of knowledge sharing outside of an organizational context. This study also offers the practical advice that participants’ knowledge sharing motivation and contribution persistence, especially newly joining members, can be increased by the inclusion of self-contribution information and conversely decreased by comparative information.