1
|
Sumner ES, DeCastro J, Costa J, Gopinath DE, Kimani E, Hakimi S, Morgan A, Best A, Nguyen H, Brooks DJ, Ul Haq B, Patrikalakis A, Yasuda H, Sieck K, Balachandran A, Chen TL, Rosman G. Personalizing driver safety interfaces via driver cognitive factors inference. Sci Rep 2024; 14:18058. [PMID: 39103366 PMCID: PMC11300826 DOI: 10.1038/s41598-024-65144-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Accepted: 06/17/2024] [Indexed: 08/07/2024] Open
Abstract
Recent advances in AI and intelligent vehicle technology hold the promise of revolutionizing mobility and transportation through advanced driver assistance systems (ADAS). Certain cognitive factors, such as impulsivity and inhibitory control have been shown to relate to risky driving behavior and on-road risk-taking. However, existing systems fail to leverage such factors in assistive driving technologies adequately. Varying the levels of these cognitive factors could influence the effectiveness and acceptance of ADAS interfaces. We demonstrate an approach for personalizing driver interaction via driver safety interfaces that are are triggered based on the inference of the driver's latent cognitive states from their driving behavior. To accomplish this, we adopt a data-driven approach and train a recurrent neural network to infer impulsivity and inhibitory control from recent driving behavior. The network is trained on a population of human drivers to infer impulsivity and inhibitory control from recent driving behavior. Using data collected from a high-fidelity vehicle motion simulator experiment, we demonstrate the ability to deduce these factors from driver behavior. We then use these inferred factors to determine instantly whether or not to engage a driver safety interface. This approach was evaluated using leave-one-out cross validation using actual human data. Our evaluations reveal that our personalized driver safety interface that captures the cognitive profile of the driver is more effective in influencing driver behavior in yellow light zones by reducing their inclination to run through them.
Collapse
Affiliation(s)
- Emily S Sumner
- Toyota Research Institute, Los Altos, CA, USA.
- , Cambridge, MA, USA.
| | | | - Jean Costa
- Toyota Research Institute, Los Altos, CA, USA
| | | | | | | | | | - Andrew Best
- Toyota Research Institute, Los Altos, CA, USA
| | - Hieu Nguyen
- Toyota Research Institute, Los Altos, CA, USA
| | - Daniel J Brooks
- Toyota Research Institute, Los Altos, CA, USA
- , Cambridge, MA, USA
| | | | | | | | - Kate Sieck
- Toyota Research Institute, Los Altos, CA, USA
| | | | | | - Guy Rosman
- Toyota Research Institute, Los Altos, CA, USA
- , Cambridge, MA, USA
| |
Collapse
|
2
|
Styler BK, Deng W, Simmons R, Admoni H, Cooper R, Ding D. Exploring Control Authority Preferences in Robotic Arm Assistance for Power Wheelchair Users. ACTUATORS 2024; 13:10.3390/act13030104. [PMID: 38586279 PMCID: PMC10996449 DOI: 10.3390/act13030104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
This paper uses mixed methods to explore the preliminary design of control authority preferences for an Assistive Robotic Manipulator (ARM). To familiarize users with an intelligent robotic arm, we perform two kitchen task iterations: one with user-initiated software autonomy (predefined autonomous actions) and one with manual control. Then, we introduce a third scenario, enabling users to choose between manual control and system delegation throughout the task. Results showed that, while manually switching modes and controlling the arm via joystick had a higher mental workload, participants still preferred full joystick control. Thematic analysis indicates manual control offered greater freedom and sense of accomplishment. Participants reacted positively to the idea of an interactive assistive system. Users did not want to ask the system to only assist, by taking over for certain actions, but also asked for situational feedback (e.g., 'How close am I (the gripper)?', 'Is the lid centered over the jug?'). This speaks to a future assistive system that ensures the user feels like they drive the system for the entirety of the task and provides action collaboration in addition to more granular situational awareness feedback.
Collapse
Affiliation(s)
- Breelyn Kane Styler
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA 15206, USA
| | - Wei Deng
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA 15206, USA
| | - Reid Simmons
- The Robotics Institute, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA
| | - Henny Admoni
- The Robotics Institute, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA
| | - Rory Cooper
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA 15206, USA
- The Robotics Institute, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA
- Department of Rehabilitation Science and Technology, University of Pittsburgh, Pittsburgh, PA 15213, USA
| | - Dan Ding
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA 15206, USA
- Department of Rehabilitation Science and Technology, University of Pittsburgh, Pittsburgh, PA 15213, USA
| |
Collapse
|
3
|
Kennedy M. The role of collaborative robotics in assistive and rehabilitation applications. Sci Robot 2023; 8:eadk6743. [PMID: 37878691 DOI: 10.1126/scirobotics.adk6743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2023]
Abstract
Collaborative robotics principles and advancements may transform the field of assistive and rehabilitation robotics.
Collapse
Affiliation(s)
- Monroe Kennedy
- Department of Mechanical Engineering, Stanford University, Stanford, CA, USA
| |
Collapse
|
4
|
Shahria MT, Sunny MSH, Zarif MII, Khan MMR, Modi PP, Ahamed SI, Rahman MH. A Novel Framework for Mixed Reality-Based Control of Collaborative Robot: Development Study. JMIR BIOMEDICAL ENGINEERING 2022; 7:e36734. [PMID: 38875679 PMCID: PMC11041473 DOI: 10.2196/36734] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Revised: 04/13/2022] [Accepted: 04/26/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Applications of robotics in daily life are becoming essential by creating new possibilities in different fields, especially in the collaborative environment. The potentials of collaborative robots are tremendous as they can work in the same workspace as humans. A framework employing a top-notch technology for collaborative robots will surely be worthwhile for further research. OBJECTIVE This study aims to present the development of a novel framework for the collaborative robot using mixed reality. METHODS The framework uses Unity and Unity Hub as a cross-platform gaming engine and project management tool to design the mixed reality interface and digital twin. It also uses the Windows Mixed Reality platform to show digital materials on holographic display and the Azure mixed reality services to capture and expose digital information. Eventually, it uses a holographic device (HoloLens 2) to execute the mixed reality-based collaborative system. RESULTS A thorough experiment was conducted to validate the novel framework for mixed reality-based control of a collaborative robot. This framework was successfully applied to implement a collaborative system using a 5-degree of freedom robot (xArm-5) in a mixed reality environment. The framework was stable and worked smoothly throughout the collaborative session. Due to the distributed nature of cloud applications, there is a negligible latency between giving a command and the execution of the physical collaborative robot. CONCLUSIONS Opportunities for collaborative robots in telerehabilitation and teleoperation are vital as in any other field. The proposed framework was successfully applied in a collaborative session, and it can also be applied in other similar potential applications for robust and more promising performance.
Collapse
Affiliation(s)
- Md Tanzil Shahria
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | - Md Samiul Haque Sunny
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | | | - Md Mahafuzur Rahaman Khan
- Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | - Preet Parag Modi
- Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| | - Sheikh Iqbal Ahamed
- Department of Computer Science, Marquette University, Milwaukee, WI, United States
| | - Mohammad H Rahman
- Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, United States
| |
Collapse
|