Assessing attentive monitoring levels in dynamic environments through visual neuro-assisted approach

Objective

This work aims to establish a framework in measuring the various attentional levels of the human operator in a real-time animated environment through a visual neuro-assisted approach.

Background

With the increasing trend of automation and remote operations, understanding human-machine interaction in dynamic environments can greatly aid to improve performance, promote operational efficiency and safety.

Method

Two independent 1-hour experiments were conducted on twenty participants where eye-tracking metrics and neuro activities from electroencephalogram (EEG) were recorded. The experiments required participants to exhibit attentive behaviour in one set and inattentive in the other. Two segments ("increasing flight numbers" and "relatively constant flight numbers") were also extracted to study the participants' visual behavioral differences in relation to aircraft numbers.

Results

For the two experimental studies, those in the attentive behavioral study show incidences of higher fixation count, fixation duration, number of aircraft spotted, and landing fixations whereas those in inattentive behavior study reveal higher zero-fixation frame count. In experiments involving 'increasing flight numbers', a higher percentage of aircraft were spotted as compared to those with 'constant flight numbers' in both the groups. Three parameters (number of aircraft spotted, and landing fixations and zero-fixation frame count) are newly established. As radar monitoring is a brain engagement activity, positive EEG data were registered in all the participants. A newly Task Engagement Index (TEI) was also formulated to predict different attentional levels.

Conclusion

Results provide a refined quantifiable tool to differentiate between attentive and inattentive monitoring behavior in a real-time dynamic environment, which can be applied across various sectors.

Recommendation

With the quantitative TEI established, this paves the way for future studies into attentional levels by regions, time based, as well as eye signature studies in relation to visual task engagement and management and determining expertise levels to be explored. Factors relating to fatigue could also be investigated using the TEI approach proposed.

Usability of cooperative surgical telemanipulation for bone milling tasks

Purpose

Cooperative surgical systems enable humans and machines to combine their individual strengths and collaborate to improve the surgical outcome. Cooperative telemanipulated systems offer the widest spectrum of cooperative functionalities, because motion scaling is possible. Haptic guidance can be used to assist surgeons and haptic feedback makes acting forces at the slave side transparent to the operator, however, overlapping and masking of forces needs to be avoided. This study evaluates the usability of a cooperative surgical telemanipulator in a laboratory setting.

Methods

Three experiments were designed and conducted for characteristic surgical task scenarios derived from field studies in orthopedics and neurosurgery to address bone tissue differentiation, guided milling and depth sensitive milling. Interaction modes were designed to ensure that no overlapping or masking of haptic guidance and haptic feedback occurs when allocating information to the haptic channel. Twenty participants were recruited to compare teleoperated modes, direct manual execution and an exemplary automated milling with respect to usability.

Results

Participants were able to differentiate compact and cancellous bone, both directly manually and teleoperatively. Both telemanipulated modes increased effectiveness measured by the mean absolute depth and contour error for guided and depth sensitive millings. Efficiency is decreased if solely a boundary constraint is used in hard material, while a trajectory guidance and manual milling perform similarly. With respect to subjective user satisfaction trajectory guidance is rated best for guided millings followed by boundary constraints and the direct manual interaction. Haptic feedback only improved subjective user satisfaction.

Conclusion

A cooperative surgical telemanipulator can improve effectiveness and efficiency close to an automated execution and enhance user satisfaction compared to direct manual interaction. At the same time, the surgeon remains part of the control loop and is able to adjust the surgical plan according to the intraoperative situation and his/her expertise at any time.

Effects of mobile augmented reality apps on impulse buying behavior: An investigation in the tourism field

Many of today's online services are designed specifically to encourage impulse buying. Moreover, many studies have shown that with the assistance of Mobile Augmented Reality, retailers have the potential to significantly improve their sales. However, the effects of Mobile AR on consumer impulse buying behavior have yet to be examined, particularly in the tourism field. Consequently, the present study integrates the Technology Acceptance Model (TAM), Stimulus-Organism-Response (SOR) framework, and flow theory to examine the effects of Mobile AR apps on tourist impulse buyingbehavior. The research model is implemented using an online questionnaire, with the results analyzed by Partial-Least-Squares Structural Equation Modeling (PLS-SEM) approach. The results obtained from 479 valid samples show that the characteristics of Mobile AR apps play an important role in governing tourist behavior in making unplanned purchases. In particular, as the utility, ease-of-use, and interactivity of the apps increase, the perceived enjoyment and satisfaction of the user also increase and give rise to a stronger impulse buying behavior. The results also reveal a mediating effect of the flow experience on the relationship between the perceived ease of use of the Mobile AR app and the user satisfaction in using the app. Overall, the findings presented in this study provide a useful source of reference for Mobile AR app developers, retailers, and tourism marketers in better understanding users' preferences for Mobile AR apps and strengthening their impulse buying behavior in the tourism context as a result.

Impact of button position and touchscreen font size on healthcare device operation by older adults

In 1993, Taiwan officially became an aging society. Degradation of physiological function during aging is inevitable; visual and physical reactions are especially vulnerable. Given the popularity of electronic devices and their vigorous development in recent years, touchscreen use is now commonplace. As society ages, many people use devices to monitor their health. Some products have gradually eschewed their traditional interfaces, which have been replaced by touchscreens. Touchscreen operation and interpretation differ between older and younger populations. Here, we focus on healthcare equipment, exploring the effects of button position and touchscreen font size on operation by older people. To understand differences between older and younger people, we invited 32 people aged 18–35 years, and 32 people aged over 65 years, to participate in our experiment. Each subject tested four button positions and four font sizes, thus 16 test interfaces in all. We found that young people found no differences among the 16 interfaces, but the older group did. Larger fonts reduced operation time for older participants. At a font size of 22 pt, the older group performed as well as the young participants. When buttons were positioned at the top of the interface, the performance of the older participants improved. Overall, use of a font size of 22 pt and top-positioned buttons optimized the performance of the older participants while use of a font size of 10 pt and bottom-positioned buttons maximally degraded their performance. Our results can be used to design interfaces appropriate for older people, thus improving their autonomy.

Selecting transferrable neurophysiological features for inter-individual emotion recognition via a shared-subspace feature elimination approach

The interplay between human emotions, personality, and motivation results in individual specificity in neurophysiological data distributions for the same emotional category. To address this issue for building an emotion recognition system based on electroencephalogram (EEG) features, we propose a shared-subspace feature elimination (SSFE) approach to identify EEG variables with common characteristics across multiple individuals. In the SSFE framework, a low-dimensional space defined by a selected number of EEG features is created to represent the inter-emotion discriminant for different pairs of subjects evaluated based on the interclass margin. Using two public databases-DEAP and MAHNOB-HCI-the performance of the SSFE is validated according to the leave-one-subject-out paradigm. The performance of the proposed framework is compared with five other feature-selection methods. The effectiveness and computational cost of the SSFE is investigated across six machine learning models based on their optimal hyperparameters. In the end, the competitive binary classification accuracy from the SSFE of arousal and valence recognitions are determined to be 0.6521 and 0.6635, respectively, for DEAP, and 0.6520 and 0.6537, respectively for MAHNOB-HCI.

Conceptual knowledge and sensitization on Asperger's syndrome based on the constructivist approach through virtual reality

The immense capabilities provided by virtual reality for enhancing empathy and altering social behaviors, make it possible to use virtual reality for the promotion of the understanding between individuals. A social issue that needs to be addressed is the discrimination and marginalization of people with Asperger's syndrome (AS) from the working force, stemming from the lack of recognition and justification of their oddities. In this paper, the effectiveness of a virtual reality simulation, raising awareness on this specific issue, is evaluated. Awareness is a term used in this research to refer to the combination of conceptual knowledge and sensitization on a specific issue. An experiment with between-group design has been carried out, where half of the participants used the simulation via a VR headset device and the other half was given a transcript of relevant medical articles instead. The comparison between the two groups indicates that the group which used the simulation gained more knowledge on AS than the group which used the transcript. Moreover, the parameter of sensitization indicates positive correlations with place illusion in the VR group.

Augmentation of haptic feedback for teleoperated robotic surgery

PURPOSE

A frequently mentioned lack of teleoperated surgical robots is the lack of haptic feedback. Haptics are not only able to mirror force information from the situs, but also to provide spatial guidance according to a surgical plan. However, superposition of the two haptic information can lead to overlapping and masking of the feedback and guidance forces. This study investigates different approaches toward a combination of both information and investigates effects on system usability.

METHODS

Preliminary studies are conducted to define parameters for two main experiments. The two main experiments constitute simulated surgical interventions where haptic guidance as well as haptic feedback provide information for the surgeon. The first main experiment considers drilling for pedicle screw placements, while the second main experiment refers to three-dimensional milling tasks such as during partial knee replacements or craniectomies. For both experiments, different guidance modes in combination with haptic feedback are evaluated regarding effectiveness (e.g., distance to target depth), efficiency and user satisfaction (e.g., detectability of discrepancies in case of technical guidance error).

RESULTS

Regarding pedicle screw placements a combination of a peripheral visual signal and a vibration constitutes a good compromise regarding distance to target depth and detectability of discrepancies. For milling tasks, trajectory guidance is able to improve efficiency and user satisfaction (e.g., perceived workload), while boundary constraints improve effectiveness. If, assistance cannot be offered in all degrees of freedom (e.g., craniectomies), a visual substitution of the haptic force feedback shows the best results, though participants prefer using haptic force feedback.

CONCLUSION

Our results suggest that in case haptic feedback and haptic assistance are combined appropriately, benefits of both haptic modalities can be exploited. Thereby, capabilities of the human-machine system are improved compared to usage of exclusively one of the haptic information.

Soft dry electroophthalmogram electrodes for human machine interaction

Current soft surface electrodes have attracted more and more attentions owing to their potential applications in biological signal monitoring, human-machine interaction (HMI) and Internet of Things (IoT). The paper presents that soft dry electrode based on polydimethylsiloxane-carbon black (PDMS-CB) conductive polymer is designed and fabricated to continuous, long-term, stable electroophthalmogram (EOG) signal recordings for HMI applications. The features corresponding to the different eye motions are extracted from the EOG data via the soft dry electrodes. Linear discriminant analysis (LDA) recognition algorithms are proposed to recognize eye motion behaviors for controlling the motion of the mobile robots. Experiment results have been demonstrated that LDA recognition algorithm achieves a relatively high recognition accuracy of 90.63% for recognizing four eye movements ('Up', 'Down', 'Right', and 'Left'). The control commands are generated with different eye motions and transmitted to the mobile robot through WiFi communication unit, which the mobile robot is successfully controlled. The soft dry electrodes have the potential in a comfortable, simple, wearable and wireless control of rehabilitation devices.

Enhanced emotional responses during social coordination with a virtual partner

Emotion and motion, though seldom studied in tandem, are complementary aspects of social experience. This study investigates variations in emotional responses during movement coordination between a human and a Virtual Partner (VP), an agent whose virtual finger movements are driven by the Haken-Kelso-Bunz (HKB) equations of Coordination Dynamics. Twenty-one subjects were instructed to coordinate finger movements with the VP in either inphase or antiphase patterns. By adjusting model parameters, we manipulated the 'intention' of VP as cooperative or competitive with the human's instructed goal. Skin potential responses (SPR) were recorded to quantify the intensity of emotional response. At the end of each trial, subjects rated the VP's intention and whether they thought their partner was another human being or a machine. We found greater emotional responses when subjects reported that their partner was human and when coordination was stable. That emotional responses are strongly influenced by dynamic features of the VP's behavior, has implications for mental health, brain disorders and the design of socially cooperative machines.