1
|
Lebrun B, Temtsin S, Vonasch A, Bartneck C. Detecting the corruption of online questionnaires by artificial intelligence. Front Robot AI 2024; 10:1277635. [PMID: 38371744 PMCID: PMC10869497 DOI: 10.3389/frobt.2023.1277635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 12/04/2023] [Indexed: 02/20/2024] Open
Abstract
Online questionnaires that use crowdsourcing platforms to recruit participants have become commonplace, due to their ease of use and low costs. Artificial intelligence (AI)-based large language models (LLMs) have made it easy for bad actors to automatically fill in online forms, including generating meaningful text for open-ended tasks. These technological advances threaten the data quality for studies that use online questionnaires. This study tested whether text generated by an AI for the purpose of an online study can be detected by both humans and automatic AI detection systems. While humans were able to correctly identify the authorship of such text above chance level (76% accuracy), their performance was still below what would be required to ensure satisfactory data quality. Researchers currently have to rely on a lack of interest among bad actors to successfully use open-ended responses as a useful tool for ensuring data quality. Automatic AI detection systems are currently completely unusable. If AI submissions of responses become too prevalent, then the costs associated with detecting fraudulent submissions will outweigh the benefits of online questionnaires. Individual attention checks will no longer be a sufficient tool to ensure good data quality. This problem can only be systematically addressed by crowdsourcing platforms. They cannot rely on automatic AI detection systems and it is unclear how they can ensure data quality for their paying clients.
Collapse
Affiliation(s)
- Benjamin Lebrun
- School of Psychology, Speech, and Hearing, University of Canterbury, Christchurch, New Zealand
| | - Sharon Temtsin
- Department of Computer Science and Software Engineering, University of Canterbury, Christchurch, New Zealand
| | - Andrew Vonasch
- School of Psychology, Speech, and Hearing, University of Canterbury, Christchurch, New Zealand
| | - Christoph Bartneck
- Department of Computer Science and Software Engineering, University of Canterbury, Christchurch, New Zealand
| |
Collapse
|
2
|
Käppler M, Mamaev I, Alagi H, Stein T, Deml B. Optimizing human-robot handovers: the impact of adaptive transport methods. Front Robot AI 2023; 10:1155143. [PMID: 37520939 PMCID: PMC10373869 DOI: 10.3389/frobt.2023.1155143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 06/27/2023] [Indexed: 08/01/2023] Open
Abstract
Humans are increasingly coming into direct physical contact with robots in the context of object handovers. The technical development of robots is progressing so that handovers can be better adapted to humans. An important criterion for successful handovers between robots and humans is the predictability of the robot for the human. The better humans can anticipate the robot's actions, the better they can adapt to them and thus achieve smoother handovers. In the context of this work, it was investigated whether a highly adaptive transport method of the object, adapted to the human hand, leads to better handovers than a non-adaptive transport method with a predefined target position. To ensure robust handovers at high repetition rates, a Franka Panda robotic arm with a gripper equipped with an Intel RealSense camera and capacitive proximity sensors in the gripper was used. To investigate the handover behavior, a study was conducted with n = 40 subjects, each performing 40 handovers in four consecutive runs. The dependent variables examined are physical handover time, early handover intervention before the robot reaches its target position, and subjects' subjective ratings. The adaptive transport method does not result in significantly higher mean physical handover times than the non-adaptive transport method. The non-adaptive transport method does not lead to a significantly earlier handover intervention in the course of the runs than the adaptive transport method. Trust in the robot and the perception of safety are rated significantly lower for the adaptive transport method than for the non-adaptive transport method. The physical handover time decreases significantly for both transport methods within the first two runs. For both transport methods, the percentage of handovers with a physical handover time between 0.1 and 0.2 s increases sharply, while the percentage of handovers with a physical handover time of >0.5 s decreases sharply. The results can be explained by theories of motor learning. From the experience of this study, an increased understanding of motor learning and adaptation in the context of human-robot interaction can be of great benefit for further technical development in robotics and for the industrial use of robots.
Collapse
Affiliation(s)
- Marco Käppler
- Institute of Human and Industrial Engineering (ifab), Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | - Ilshat Mamaev
- Intelligent Process Automation and Robotics Lab (IAR–IPR), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | - Hosam Alagi
- Intelligent Process Automation and Robotics Lab (IAR–IPR), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | - Thorsten Stein
- BioMotion Center, Institute of Sports and Sports Science (IfSS), Karlsruhe Institute of Technology, Karlsruhe, Germany
| | - Barbara Deml
- Institute of Human and Industrial Engineering (ifab), Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| |
Collapse
|
3
|
Doherty EJ, Spencer CA, Burnison J, Čeko M, Chin J, Eloy L, Haring K, Kim P, Pittman D, Powers S, Pugh SL, Roumis D, Stephens JA, Yeh T, Hirshfield L. Interdisciplinary views of fNIRS: Current advancements, equity challenges, and an agenda for future needs of a diverse fNIRS research community. Front Integr Neurosci 2023; 17:1059679. [PMID: 36922983 PMCID: PMC10010439 DOI: 10.3389/fnint.2023.1059679] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Accepted: 02/08/2023] [Indexed: 03/02/2023] Open
Abstract
Functional Near-Infrared Spectroscopy (fNIRS) is an innovative and promising neuroimaging modality for studying brain activity in real-world environments. While fNIRS has seen rapid advancements in hardware, software, and research applications since its emergence nearly 30 years ago, limitations still exist regarding all three areas, where existing practices contribute to greater bias within the neuroscience research community. We spotlight fNIRS through the lens of different end-application users, including the unique perspective of a fNIRS manufacturer, and report the challenges of using this technology across several research disciplines and populations. Through the review of different research domains where fNIRS is utilized, we identify and address the presence of bias, specifically due to the restraints of current fNIRS technology, limited diversity among sample populations, and the societal prejudice that infiltrates today's research. Finally, we provide resources for minimizing bias in neuroscience research and an application agenda for the future use of fNIRS that is equitable, diverse, and inclusive.
Collapse
Affiliation(s)
- Emily J. Doherty
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO, United States
| | - Cara A. Spencer
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO, United States
| | | | - Marta Čeko
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO, United States
| | - Jenna Chin
- College of Arts, Humanities, and Social Sciences, Psychology, University of Denver, Denver, CO, United States
| | - Lucca Eloy
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO, United States
| | - Kerstin Haring
- Department of Computer Science, University of Denver, Denver, CO, United States
| | - Pilyoung Kim
- College of Arts, Humanities, and Social Sciences, Psychology, University of Denver, Denver, CO, United States
| | - Daniel Pittman
- Department of Computer Science, University of Denver, Denver, CO, United States
| | - Shannon Powers
- College of Arts, Humanities, and Social Sciences, Psychology, University of Denver, Denver, CO, United States
| | - Samuel L. Pugh
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO, United States
| | | | - Jaclyn A. Stephens
- Department of Occupational Therapy, Colorado State University, Fort Collins, CO, United States
| | - Tom Yeh
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
| | - Leanne Hirshfield
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
- Institute of Cognitive Science, University of Colorado Boulder, Boulder, CO, United States
| |
Collapse
|