|
2
|
Iwanaga T, Wang HH, Hamilton SH, Grimm V, Koralewski TE, Salado A, Elsawah S, Razavi S, Yang J, Glynn P, Badham J, Voinov A, Chen M, Grant WE, Peterson TR, Frank K, Shenk G, Barton CM, Jakeman AJ, Little JC. Socio-technical scales in socio-environmental modeling: Managing a system-of-systems modeling approach. ENVIRONMENTAL MODELLING & SOFTWARE : WITH ENVIRONMENT DATA NEWS 2021; 135:104885. [PMID: 33041631 PMCID: PMC7537632 DOI: 10.1016/j.envsoft.2020.104885] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 09/29/2020] [Indexed: 05/05/2023]
Abstract
System-of-systems approaches for integrated assessments have become prevalent in recent years. Such approaches integrate a variety of models from different disciplines and modeling paradigms to represent a socio-environmental (or social-ecological) system aiming to holistically inform policy and decision-making processes. Central to the system-of-systems approaches is the representation of systems in a multi-tier framework with nested scales. Current modeling paradigms, however, have disciplinary-specific lineage, leading to inconsistencies in the conceptualization and integration of socio-environmental systems. In this paper, a multidisciplinary team of researchers, from engineering, natural and social sciences, have come together to detail socio-technical practices and challenges that arise in the consideration of scale throughout the socio-environmental modeling process. We identify key paths forward, focused on explicit consideration of scale and uncertainty, strengthening interdisciplinary communication, and improvement of the documentation process. We call for a grand vision (and commensurate funding) for holistic system-of-systems research that engages researchers, stakeholders, and policy makers in a multi-tiered process for the co-creation of knowledge and solutions to major socio-environmental problems.
Collapse
Affiliation(s)
- Takuya Iwanaga
- Institute for Water Futures and Fenner School of Environment and Society, The Australian National University, Canberra, Australia
| | - Hsiao-Hsuan Wang
- Ecological Systems Laboratory, Department of Ecology and Conservation Biology, Texas A&M University, College Station, TX, 77843, USA
| | - Serena H Hamilton
- Institute for Water Futures and Fenner School of Environment and Society, The Australian National University, Canberra, Australia
- CSIRO Land & Water, Canberra, Australia
| | - Volker Grimm
- Helmholtz Centre for Environmental Research - UFZ, Department of Ecological Modelling, Leipzig, Germany
- University of Potsdam, Plant Ecology and Nature Conservation, Potsdam, Germany
| | - Tomasz E Koralewski
- Ecological Systems Laboratory, Department of Ecology and Conservation Biology, Texas A&M University, College Station, TX, 77843, USA
| | - Alejandro Salado
- Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, 24061, USA
| | - Sondoss Elsawah
- Institute for Water Futures and Fenner School of Environment and Society, The Australian National University, Canberra, Australia
- School of Electrical Engineering and Information Technology, University of New South Wales, Australian Defence Force Academy, Canberra, ACT, Australia
| | - Saman Razavi
- Global Institute for Water Security, School of Environment and Sustainability, Department of Civil, Geological, and Environmental Engineering, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | - Jing Yang
- National Institute of Water and Atmospheric Research, New Zealand
| | - Pierre Glynn
- U.S. Department of the Interior, U.S. Geological Survey, Reston, VA, USA
| | - Jennifer Badham
- Centre for Research in Social Simulation, University of Surrey, Guildford, GU2 7XH, United Kingdom
| | - Alexey Voinov
- Center on Persuasive Systems for Wise Adaptive Living (PERSWADE), Faculty of Engineering & IT, University of Technology, Sydney, Australia
- Faculty of Engineering Technology, University of Twente, Netherlands
| | - Min Chen
- Key Laboratory of Virtual Geographic Environment (Ministry of Education of PRC), Nanjing Normal University, Nanjing, 210023, China
| | - William E Grant
- Ecological Systems Laboratory, Department of Ecology and Conservation Biology, Texas A&M University, College Station, TX, 77843, USA
| | - Tarla Rai Peterson
- Environmental Science and Engineering Program, University of Texas at El Paso, El Paso, TX, 79968, USA
| | - Karin Frank
- Helmholtz Centre for Environmental Research - UFZ, Department of Ecological Modelling, Leipzig, Germany
| | - Gary Shenk
- U.S Geological Survey, Chesapeake Bay Program, Annapolis, MD, 21403, USA
| | - C Michael Barton
- Center for Social Dynamics & Complexity, School of Human Evolution & Social Change, Arizona State University, Tempe, AZ, USA
| | - Anthony J Jakeman
- Institute for Water Futures and Fenner School of Environment and Society, The Australian National University, Canberra, Australia
| | - John C Little
- Department of Civil and Environmental Engineering, Virginia Tech, Blacksburg, VA, 24061, USA
| |
Collapse
|
|
3
|
Abstract
Background: Evaluation of the quality of research software is a challenging and relevant issue, still not sufficiently addressed by the scientific community. Methods: Our contribution begins by defining, precisely but widely enough, the notions of research software and of its authors followed by a study of the evaluation issues, as the basis for the proposition of a sound assessment protocol: the CDUR procedure. Results: CDUR comprises four steps introduced as follows: Citation, to deal with correct RS identification, Dissemination, to measure good dissemination practices, Use, devoted to the evaluation of usability aspects, and Research, to assess the impact of the scientific work. Conclusions: Some conclusions and recommendations are finally included. The evaluation of research is the keystone to boost the evolution of the Open Science policies and practices. It is as well our belief that research software evaluation is a fundamental step to induce better research software practices and, thus, a step towards more efficient science.
Collapse
Affiliation(s)
- Teresa Gomez-Diaz
- Laboratoire d'Informatique Gaspard-Monge, Centre National de la Recherche Scientifique, University of Paris-Est Marne-la-Vallée, Marne-la-Vallée, France
| | | |
Collapse
|
|
4
|
Abstract
Background: Evaluation of the quality of research software is a challenging and relevant issue, still not sufficiently addressed by the scientific community. Methods: Our contribution begins by defining, precisely but widely enough, the notions of research software and of its authors followed by a study of the evaluation issues, as the basis for the proposition of a sound assessment protocol: the CDUR procedure. Results: CDUR comprises four steps introduced as follows: Citation, to deal with correct RS identification, Dissemination, to measure good dissemination practices, Use, devoted to the evaluation of usability aspects, and Research, to assess the impact of the scientific work. Conclusions: Some conclusions and recommendations are finally included. The evaluation of research is the keystone to boost the evolution of the Open Science policies and practices. It is as well our belief that research software evaluation is a fundamental step to induce better research software practices and, thus, a step towards more efficient science.
Collapse
Affiliation(s)
- Teresa Gomez-Diaz
- Laboratoire d'Informatique Gaspard-Monge, Centre National de la Recherche Scientifique, University of Paris-Est Marne-la-Vallée, Marne-la-Vallée, France
| | | |
Collapse
|
|
6
|
Kanewala U, Bieman JM. Testing Scientific Software: A Systematic Literature Review. INFORMATION AND SOFTWARE TECHNOLOGY 2014; 56:1219-1232. [PMID: 25125798 PMCID: PMC4128280 DOI: 10.1016/j.infsof.2014.05.006] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
CONTEXT Scientific software plays an important role in critical decision making, for example making weather predictions based on climate models, and computation of evidence for research publications. Recently, scientists have had to retract publications due to errors caused by software faults. Systematic testing can identify such faults in code. OBJECTIVE This study aims to identify specific challenges, proposed solutions, and unsolved problems faced when testing scientific software. METHOD We conducted a systematic literature survey to identify and analyze relevant literature. We identified 62 studies that provided relevant information about testing scientific software. RESULTS We found that challenges faced when testing scientific software fall into two main categories: (1) testing challenges that occur due to characteristics of scientific software such as oracle problems and (2) testing challenges that occur due to cultural differences between scientists and the software engineering community such as viewing the code and the model that it implements as inseparable entities. In addition, we identified methods to potentially overcome these challenges and their limitations. Finally we describe unsolved challenges and how software engineering researchers and practitioners can help to overcome them. CONCLUSIONS Scientific software presents special challenges for testing. Specifically, cultural differences between scientist developers and software engineers, along with the characteristics of the scientific software make testing more difficult. Existing techniques such as code clone detection can help to improve the testing process. Software engineers should consider special challenges posed by scientific software such as oracle problems when developing testing techniques.
Collapse
|