An Adaptive Human-Robotic Interaction Architecture for Augmenting Surgery Performance Using Real-Time Workload Sensing-Demonstration of a Semi-autonomous Suction Tool

OBJECTIVE

This study developed and evaluated a mental workload-based adaptive automation (MWL-AA) that monitors surgeon cognitive load and assist during cognitively demanding tasks and assists surgeons in robotic-assisted surgery (RAS).

BACKGROUND

The introduction of RAS makes operators overwhelmed. The need for precise, continuous assessment of human mental workload (MWL) states is important to identify when the interventions should be delivered to moderate operators' MWL.

METHOD

The MWL-AA presented in this study was a semi-autonomous suction tool. The first experiment recruited ten participants to perform surgical tasks under different MWL levels. The physiological responses were captured and used to develop a real-time multi-sensing model for MWL detection. The second experiment evaluated the effectiveness of the MWL-AA, where nine brand-new surgical trainees performed the surgical task with and without the MWL-AA. Mixed effect models were used to compare task performance, objective- and subjective-measured MWL.

RESULTS

The proposed system predicted high MWL hemorrhage conditions with an accuracy of 77.9%. For the MWL-AA evaluation, the surgeons' gaze behaviors and brain activities suggested lower perceived MWL with MWL-AA than without. This was further supported by lower self-reported MWL and better task performance in the task condition with MWL-AA.

CONCLUSION

A MWL-AA systems can reduce surgeons' workload and improve performance in a high-stress hemorrhaging scenario. Findings highlight the potential of utilizing MWL-AA to enhance the collaboration between the autonomous system and surgeons. Developing a robust and personalized MWL-AA is the first step that can be used do develop additional use cases in future studies.

APPLICATION

The proposed framework can be expanded and applied to more complex environments to improve human-robot collaboration.

Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training

OBJECTIVE

The aim of this study is to assess the relationship between eye-tracking measures and perceived workload in robotic surgical tasks.

BACKGROUND

Robotic techniques provide improved dexterity, stereoscopic vision, and ergonomic control system over laparoscopic surgery, but the complexity of the interfaces and operations may pose new challenges to surgeons and compromise patient safety. Limited studies have objectively quantified workload and its impact on performance in robotic surgery. Although not yet implemented in robotic surgery, minimally intrusive and continuous eye-tracking metrics have been shown to be sensitive to changes in workload in other domains.

METHODS

Eight surgical trainees participated in 15 robotic skills simulation sessions. In each session, participants performed up to 12 simulated exercises. Correlation and mixed-effects analyses were conducted to explore the relationships between eye-tracking metrics and perceived workload. Machine learning classifiers were used to determine the sensitivity of differentiating between low and high workload with eye-tracking features.

RESULTS

Gaze entropy increased as perceived workload increased, with a correlation of .51. Pupil diameter and gaze entropy distinguished differences in workload between task difficulty levels, and both metrics increased as task level difficulty increased. The classification model using eye-tracking features achieved an accuracy of 84.7% in predicting workload levels.

CONCLUSION

Eye-tracking measures can detect perceived workload during robotic tasks. They can potentially be used to identify task contributors to high workload and provide measures for robotic surgery training.

APPLICATION

Workload assessment can be used for real-time monitoring of workload in robotic surgical training and provide assessments for performance and learning.

Gaze and Eye Tracking: Techniques and Applications in ADAS

Tracking drivers’ eyes and gazes is a topic of great interest in the research of advanced driving assistance systems (ADAS). It is especially a matter of serious discussion among the road safety researchers’ community, as visual distraction is considered among the major causes of road accidents. In this paper, techniques for eye and gaze tracking are first comprehensively reviewed while discussing their major categories. The advantages and limitations of each category are explained with respect to their requirements and practical uses. In another section of the paper, the applications of eyes and gaze tracking systems in ADAS are discussed. The process of acquisition of driver’s eyes and gaze data and the algorithms used to process this data are explained. It is explained how the data related to a driver’s eyes and gaze can be used in ADAS to reduce the losses associated with road accidents occurring due to visual distraction of the driver. A discussion on the required features of current and future eye and gaze trackers is also presented.

Modeling take-over performance in level 3 conditionally automated vehicles

Taking over vehicle control from a Level 3 conditionally automated vehicle can be a demanding task for a driver. The take-over determines the controllability of automated vehicle functions and thereby also traffic safety. This paper presents models predicting the main take-over performance variables take-over time, minimum time-to-collision, brake application and crash probability. These variables are considered in relation to the situational and driver-related factors time-budget, traffic density, non-driving-related task, repetition, the current lane and driver's age. Regression models were developed using 753 take-over situations recorded in a series of driving simulator experiments. The models were validated with data from five other driving simulator experiments of mostly unrelated authors with another 729 take-over situations. The models accurately captured take-over time, time-to-collision and crash probability, and moderately predicted the brake application. Especially the time-budget, traffic density and the repetition strongly influenced the take-over performance, while the non-driving-related tasks, the lane and drivers' age explained a minor portion of the variance in the take-over performances.

Multi-modal demands of a smartphone used to place calls and enter addresses during highway driving relative to two embedded systems

There is limited research on trade-offs in demand between manual and voice interfaces of embedded and portable technologies. Mehler et al. identified differences in driving performance, visual engagement and workload between two contrasting embedded vehicle system designs (Chevrolet MyLink and Volvo Sensus). The current study extends this work by comparing these embedded systems with a smartphone (Samsung Galaxy S4). None of the voice interfaces eliminated visual demand. Relative to placing calls manually, both embedded voice interfaces resulted in less eyes-off-road time than the smartphone. Errors were most frequent when calling contacts using the smartphone. The smartphone and MyLink allowed addresses to be entered using compound voice commands resulting in shorter eyes-off-road time compared with the menu-based Sensus but with many more errors. Driving performance and physiological measures indicated increased demand when performing secondary tasks relative to 'just driving', but were not significantly different between the smartphone and embedded systems. Practitioner Summary: The findings show that embedded system and portable device voice interfaces place fewer visual demands on the driver than manual interfaces, but they also underscore how differences in system designs can significantly affect not only the demands placed on drivers, but also the successful completion of tasks.

Bayesian Highest Density Intervals of Take-Over Times for Highly Automated Driving in Different Traffic Densities

Current human factors research on automated driving aims to ensure its safe introduction into road traffic. Although informative results are crucial for this purpose, most studies rely on point estimates and dichotomous reject-nonreject decisions that have been declared obsolete by more recent statistical approaches like new statistics (Cumming, 2014) or Bayesian parameter estimation (Kruschke, 2015). In this work, we show the objective advantages of Bayesian parameter estimation and demonstrate its abundance of information on parameters. In Study 1, we estimate take-over times with a relatively uninformed prior distribution. In Study 2, the resulting posterior distributions of Study 1 were then used as informed prior distributions for interval estimations of mean, standard deviation and distribution shape of take-over time in different traffic densities. We obtained 95 % credible interval widths between 490 ms and 600 ms for mean take-over times, depending on the condition. These intervals include the 95 % most probable values of the mean take-over time and represent a quantification of uncertainty in the estimation. Given the data and the experimental conditions, a take-over requires most likely 2.51 seconds [2.27, 2.76] when there is no traffic, 3.40 seconds [3.11, 3.71] in medium traffic and 3.50 seconds [3.21, 3.78] in high traffic. Bayesian model comparison with Bayes Factor is discussed as an alternative approach in conclusion.

Leading to distraction: Driver distraction, lead car, and road environment

Driver distraction is strongly associated with crashes and near-misses, and despite the attention this topic has received in recent years, the effect of different types of distracting task on driving performance remains unclear. In the case of non-visual distractions, such as talking on the phone or other engaging verbal tasks that do not require a visual input, a common finding is reduced lateral variability in steering and gaze patterns where participants concentrate their gaze towards the centre of the road and their steering control is less variable. In the experiments presented here, we examined whether this finding is more pronounced in the presence of a lead car (which may provide a focus point for gaze) and whether the behaviour of the lead car has any influence on the driver's steering control. In addition, both visual and non-visual distraction tasks were used, and their effect on different road environments (straight and curved roadways) was assessed. Visual distraction was found to increase variability in both gaze patterns and steering control, non-visual distraction reduced gaze and steering variability in conditions without a lead car; in the conditions where a lead car was present there was no significant difference from baseline. The lateral behaviour of the lead car did not have an effect on steering performance, a finding which indicates that a lead car may not necessarily be used as an information point. Finally, the effects of driver distraction were different for straight and curved roadways, indicating a stronger influence of the road environment in steering than previously thought.

How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving

Highly automated driving constitutes a temporary transfer of the primary driving task from the driver to the automated vehicle. In case of system limits, drivers take back control of the vehicle. This study investigates the effect of varying traffic situations and non-driving related tasks on the take-over process and quality. The experiment is conducted in a high-fidelity driving simulator. The standardized visual Surrogate Reference Task (SuRT) and the cognitive n-back Task are used to simulate the non-driving related tasks. Participants experience four different traffic situations. Results of this experiment show a strong influence of the traffic situations on the take-over quality in a highway setting, if the traffic density is high. The non-driving related tasks SuRT and the n-back Task show similar effects on the take-over process with a higher total number of collisions by the SuRT in the high density traffic situation.

A field study on the impact of variations in shortterm memory demands on drivers' visual attention and driving performance across three age groups

OBJECTIVE

The aim of this study was to assess sensitivity of visual attention and driving performance for detecting changes in driver cognitive workload across different age groups.

BACKGROUND

The literature shows mixed results concerning the sensitivity of gaze concentration metrics to variations in cognitive demand. No studies appear showing how age affects gaze allocation during cognitive demand.

METHOD

Recordings of drivers' gaze and driving performance by individuals in their 20s, 40s, and 60s were captured in actual driving conditions during three levels of cognitive demand.

RESULTS

Gaze concentration increased with task difficulty through the low and moderate levels of demand and then appeared to level out at the high demand level. At the moderate difficulty level, gaze concentration increased by 2.4 cm (approximately 2 degrees) from the reference period. The degree of gaze concentration with added cognitive demand is not related to age in the relatively healthy drivers studied. Driving performance measures did not show a consistent relationship with the objective demand level.

CONCLUSION

Gaze concentration appears at low levels of cognitive demand prior to the appearance of marked decrements in driving control. There is no compelling evidence from this study that driving performance measures can be used to index differences in workload prior to capacity saturation.

APPLICATION

Drivers' awareness of vehicle surroundings is incrementally affected by increases in cognitive demand. Developers of more advanced driver support systems should consider gaze concentration as a measure of driver cognitive workload. This recommendation is particularly relevant in light of the added benefits of gaze measurements for detecting visual demand.

The effects of lisdexamfetamine dimesylate on the driving performance of young adults with ADHD: a randomized, double-blind, placebo-controlled study using a validated driving simulator paradigm

Young adults with Attention Deficit Hyperactivity Disorder (ADHD) have been shown to be at increased risk for impairment in driving behaviors. While stimulant medications have proven efficacy in reducing ADHD symptomatology, there is limited knowledge as to their effects on driving impairment. The main aim of this study was to assess the impact of lisdexamfetamine dimesylate (LDX) on driving performance in young adults with ADHD using a validated driving simulation paradigm. This was a randomized, double-blind, 6-week, placebo-controlled, parallel-design study of LDX vs. a placebo on driving performance in a validated driving simulation paradigm. Subjects were sixty-one outpatients of both sexes, 18-26 years of age, who met DSM-IV criteria for ADHD. Subjects were randomized to receive LDX or placebo after a baseline driving simulation and completed a second driving simulation six weeks after beginning drug or placebo. Examination of reaction time across five surprise events at post-treatment showed a significant positive effect of medication status. LDX treatment was also associated with significantly fewer accidents vs. placebo. LDX treatment was associated with significantly faster reaction times and a lower rate of simulated driving collisions than placebo. These results suggest that LDX may reduce driving risks in young adults with ADHD.