Supporting interruption management and multimodal interface design: three meta-analyses of task performance as a function of interrupting task modality

Task-Relevant Smartphone Messages Within Work Zones: A Driving Simulation Study

OBJECTIVE

This study explored the impact of in-vehicle messages relative to roadside messages to alert drivers to events within a simulated work zone, in order to determine if these messages can improve driving performance within the work zone.

BACKGROUND

Safety risks in work zones are usually mitigated by design standards and clear signage to communicate work zone information to drivers. Due to distraction and other driving task demands, these signs are not always noticed by motorists, nor are they always followed when they are noticed.

METHOD

The driving simulation tested drivers in two different types of work zones, shoulder work, and lane closure. Participants drove through these work zones three times, each with different messaging interfaces to communicate hazardous events to the driver. The interfaces included a roadside, portable changeable message sign, a smartphone presenting only auditory messages, and a smartphone presenting audio-visual messages.

RESULTS

There was significantly better driving performance on key metrics including lane deviation for the in-vehicle message conditions relative to the roadside signs. Furthermore, drivers directed visual attention toward the roadway for the in-vehicle message conditions relative to the roadside sign condition.

CONCLUSION

The results indicate that in-vehicle messaging could provide benefits to primary task performance in driving if the message content is appropriately designed.

APPLICATION

The findings provide support for a design framework to support in-vehicle communication to drivers approaching work zones and other environments to safely alert them to hazards.

Do human work interruptions bring positive affective responses?-Based on the mediation of coping potential and the moderation of polychronicity

This research aims to investigate the impact of human work interruptions on positive affective responses and their underlying mechanisms in the Chinese context. In the first stage, this study conducted face-to-face semi-structured interviews with 29 employees representing diverse industries. The grounded theory research method was used to extract the construct of human work interruption, identify its core attributes, and capture the naturally emerging storyline of "human work interruptions - coping potential - polychronicity - positive affective responses". In the second stage, a theoretical model was constructed and validated using 362 questionnaires. The results indicate that in the Chinese context: (1) human work interruptions can trigger positive affective responses; (2) coping potential mediates the relationship between human work interruptions and positive affective responses; (3) when individuals have a higher level of polychronicity, the impact of human work interruptions on positive affective responses via coping potential is enhanced. The findings of this study effectively address the hypothesis of the "positive aspect" of work interruptions proposed by management scholars and contribute to the existing literature on work interruptions and positive affective responses. Moreover, this research provides practical and theoretical implications for managers and employees in managing and coping with human work interruptions.

The Role of Visual and Auditory Communication in the Performance of a Joint Team Task

OBJECTIVE

This study examines visual, auditory, and the combination of both (bimodal) coupling modes in the performance of a two-person perceptual-motor task, in which one person provides the perceptual inputs and the other the motor inputs.

BACKGROUND

Parking a plane or landing a helicopter on a mountain top requires one person to provide motor inputs while another person provides perceptual inputs. Perceptual inputs are communicated either visually, auditorily, or through both cues.

METHODS

One participant drove a remote-controlled car around an obstacle and through a target, while another participant provided auditory, visual, or bimodal cues for steering and acceleration. Difficulty was manipulated using target size. Performance (trial time, path variability), cue rate, and spatial ability were measured.

RESULTS

Visual coupling outperformed auditory coupling. Bimodal performance was best in the most difficult task condition but also high in the easiest condition. Cue rate predicted performance in all coupling modes. Drivers with lower spatial ability required a faster auditory cue rate, whereas drivers with higher ability performed best with a lower rate.

CONCLUSION

Visual cues result in better performance when only one coupling mode is available. As predicted by multiple resource theory, when both cues are available, performance depends more on auditory cueing. In particular, drivers must be able to transform auditory cues into spatial actions.

APPLICATION

Spotters should be trained to provide an appropriate cue rate to match the spatial ability of the driver or pilot. Auditory cues can enhance visual communication when the interpersonal task is visual with spatial outputs.

Multimodal interaction: Input-output modality combinations for identification tasks in augmented reality

Multimodal interaction (MMI) is being widely implemented, especially in new technologies such as augmented reality (AR) systems since it is presumed to support a more natural, efficient, and flexible form of interaction. However, limited research has been done to investigate the proper application of MMI in AR. More specifically, the effects of combining different input and output modalities during MMI in AR are still not fully understood. Therefore, this study aims to examine the independent and combined effects of different input and output modalities during a typical AR task. 20 young adults participated in a controlled experiment in which they were asked to perform a simple identification task using an AR device in different input (speech, gesture, multimodal) and output (VV-VA, VV-NA, NV-VA, NV-NA) conditions. Results showed that there were differences in the influence of input and output modalities on task performance, workload, perceived appropriateness, and user preference. Interaction effects between the input and output conditions on the performance metrics were also evident in this study, suggesting that although multimodal input is generally preferred by the users, it should be implemented with caution since its effectiveness is highly influenced by the processing code of the system output. This study, which is the first of its kind, has revealed several new implications regarding the application of MMI in AR systems.

When to use vibrotactile displays? A meta-analysis for the role of vibrotactile displays in human-computer interaction

OBJECTIVE

This study aims to investigate the benefits of unimodal tactile displays relative to other modal displays and the performance gains of adding redundant tactile displays by integrating empirical studies.

BACKGROUND

Tactile displays have attracted increasing attention in recent years due to their unique advantages. Synthesizing experimental data is necessary to analyze the performance benefits of tactile displays for participants and better help practitioners in utilizing them.

METHOD

Five meta-analyses were conducted. Two meta-analyses compared the participants' performance between tactile and other modal displays (visual vs. tactile and auditory vs. tactile). Three meta-analyses examined the performance gains of adding redundant tactile displays based on other modal displays (visual vs. visual + tactile, auditory vs. auditory + tactile, and visual + auditory vs. visual + auditory + tactile). The related moderator variables, the types of presented information and concurrent tasks, were analyzed.

RESULTS

Little evidence shows the performance difference between tactile and auditory displays. Tactile displays are more beneficial than visual displays for presenting alert information or in the situation with a visual concurrent task. The performance gains of adding redundant tactile displays to other modal displays also depend on the specific type of presented information and the concurrent task.

CONCLUSION

When using tactile displays to convey information, interface designers should consider the specific type of presented information and the concurrent tasks.

APPLICATIONS

The present study's findings can provide some implications for designers to utilize tactile displays when they construct and implement information displays.

Effectiveness of Lateral Auditory Collision Warnings: Should Warnings Be Toward Danger or Toward Safety?

OBJECTIVE

The present study investigated the design of spatially oriented auditory collision-warning signals to facilitate drivers' responses to potential collisions.

BACKGROUND

Prior studies on collision warnings have mostly focused on manual driving. It is necessary to examine the design of collision warnings for safe takeover actions in semi-autonomous driving.

METHOD

In a video-based semi-autonomous driving scenario, participants responded to pedestrians walking across the road, with a warning tone presented in either the avoidance direction or the collision direction. The time interval between the warning tone and the potential collision was also manipulated. In Experiment 1, pedestrians always started walking from one side of the road to the other side. In Experiment 2, pedestrians appeared in the middle of the road and walked toward either side of the road.

RESULTS

In Experiment 1, drivers reacted to the pedestrian faster with collision-direction warnings than with avoidance-direction warnings. In Experiment 2, the difference between the two warning directions became nonsignificant. In both experiments, shorter time intervals to potential collisions resulted in faster reactions but did not influence the effect of warning direction.

CONCLUSION

The collision-direction warnings were advantageous over the avoidance-direction warnings only when they occurred at the same lateral location as the pedestrian, indicating that this advantage was due to the capture of attention by the auditory warning signals.

APPLICATION

The present results indicate that drivers would benefit most when warnings occur at the side of potential collision objects rather than the direction of a desirable action during semi-autonomous driving.

Takeover requests for automated driving: The effects of signal direction, lead time, and modality on takeover performance

Vehicle-to-driver takeover will still be needed in semi-autonomous vehicles. Due to the complexity of the takeover process, it is important to develop interfaces to support good takeover performance. Multimodal displays have been proposed as a candidate for the design of takeover requests (TORs), but many questions remain unanswered regarding the effectiveness of this approach. This study investigated the effects of takeover signal direction (ipsilateral vs. contralateral), lead time (4 vs. 7 s), and modality (uni-, bi-, and trimodal combinations of visual, auditory, and tactile signals) on automated vehicle takeover performance. Twenty-four participants rode in a simulated SAE Level 3 vehicle and performed a series of takeover tasks when presented with a TOR. Overall, single and multimodal signals with a tactile component were correlated with the faster takeover and information processing times, and were perceived as most useful. Ipsilateral signals showed a marginally significant benefit to takeover times compared to contralateral signals. Finally, a shorter lead time was associated with faster takeover times, but also poorer takeover quality. Findings from this study can inform the design of in-vehicle information and warning systems for next-generation transportation.

Simple auditory and visual interruptions of a continuous visual tracking task: modality effects and time course of interference

Research has produced conflicting evidence regarding whether performance of an on-going visual task is disrupted more by an interruption from a visual or an auditory alert. Tasks and alerts studied to date have been complex or idiosyncratic. This experiment examined how the modality of simple alerts-visual icons or auditory tones-affected performance of an on-going visual task. Participants (58 females and 4 males) tracked a visual target while performing a choice reaction time task in response to alerts. Visual alerts were more harmful to performance of the tracking task. Dual task workload was lowest with an auditory alert, provided there was not noise present. Interruptions affected tracking task performance for around 1500 ms. Results supported the predictions of Multiple Resources Theory and showed no evidence of auditory preemption. In practical applications for which an on-going visual task is interrupted, auditory alerts may be less disruptive and may reduce perceived workload. Practitioner Summary: Many practical scenarios involve on-going visual tasks that are interrupted by simple alerts requiring a simple response. Auditory alerts may be less disruptive than visual alerts and may reduce perceived workload. A conservative estimate is that the effects of even simple interruptions will last a minimum of 1500 ms. Abbreviations: ANOVA: analysis of variance; LSD: least significant difference; TLX: task load index.

Shaping driver-vehicle interaction in autonomous vehicles: How the new in-vehicle systems match the human needs

Autonomous vehicle (AV) technology has brought a shift in the traditional role of the driver. This paper applies a user-centred design approach to designing a new AV interior to better support drivers. Three empirical studies were conducted, involving a total of 92 drivers (with 44 in Study 1, 12 in Study 2, and 36 in Study 3) to explore user needs and requirements in an AV. In Study 1, safety and comfort, together with a variety of non-driving activities, were identified as the principal concerns about future autonomous vehicles. Based on these findings, Study 2 proposes a new rotatable seating position for AVs, with an in-vehicle information display to facilitate users' activities and situational awareness while driving. Study 3 consists of a series of laboratory simulator evaluation studies, and this indicated that drivers in the proposed design condition had better situational awareness in an AV when dealing with take-over situations. Such findings suggest the possibility of applying rear-facing seats in autonomous vehicles to support in-vehicle non-driving activities. Some specific implications of designs to enhance a driver's situational awareness have also been suggested.

Using a ripple wall to help blind people measure the water level in a container

The aim of this study is to determine whether swiping the ripple wall of a container can help blind people to measure the water level in it. Swiping the ripples on the wall of a container above the water level produces a different sound from doing so below the water level, and this difference in sound may be able to indicate the level of water in the container. Such sound differences associated with 27 3 D-printed containers with a capacity of 500 ml and various forms were recorded. One of the printed containers and a commercially available beverage container were tested by blind people to measure water levels in three operations. The experimental results reveal that the thickness of the wall affected the sound most strongly. The errors in the estimated water levels were significantly smaller when the containers was lifted and swiped than when it was lifted only. Practitioner summary: Lifting only is used by blind people to judge the fullness of a container. The experimental results reveal that the errors in the estimated water levels were significantly smaller when blind people lifted and swiped a 500 ml container with a ripple wall than when it was lifted only. Abbreviations: FA I: fast adapting fibers I; FA II: fast adapting fibers II; SA I: slowly adapting fibers I; SA II: slowly adapting fibers II.

Workflow disruptions and provider situation awareness in acute care: An observational study with emergency department physicians and nurses

BACKGROUND

The fast-paced and rapidly changing environment of an Emergency Department (ED) requires providers to have a high level of situation awareness (SA). However, acute clinical care also encompasses a multitude of interruption-laden work processes that might degrade SA. It is therefore important to understand how frequent interruptions affect ED provider cognition in general and SA in particular.

OBJECTIVE

We aimed to examine how sources and contents of provider workflow interruptions influence situation awareness of ED physicians and nurses.

METHODS

This prospective, multi-method study combined standardized observations, self-reports of ED providers, and ED administrative data of staffing and patient load. Expert observers identified ED providers' workflow interruptions during 90min observation sessions. Afterwards, each provider reported perceived disruptiveness and situation awareness. Controlling for patient load, patient acuity and staffing, we conducted regression analyses to explore prospective associations between interruptions and provider outcomes.

RESULTS

During 74 observation sessions of overall 110h and 40min, we observed 1205 workflow interruptions (mean rate: 10.9 interruptions/hour). Provider situation awareness was fairly high (M = 7.10; scale 0-10) with no difference between ED physicians and nurses. After controlling for ED workload data, we observed that high rates of interruptions were associated with lower levels of situation awareness (β = -0.27). Further analyses revealed that particularly interruptions by telephone/beeper, technical malfunctions as well as interruptive communication related to completed cases were correlated to low SA.

DISCUSSION

This study in a naturalistic ED setting shows that ED physicians and nurses continuously cope with disruptions and interruptions. Our findings reveal that highly interruptive workflow environments impede providers' situation awareness. Moreover, it sheds light on specific sources and contents of interruptions that influence providers' SA in acute care.

CONCLUSION

Frequent workflow interruptions can degrade ED providers' situation awareness. A deeper understanding of how avoidable and unavoidable interruptions affect provider cognitions with particular focus on social and technology-related disruptions is required. Further emphasis should be placed on the effective application of work re-design in this context to foster safe and efficient patient care.

Mid-Air Tactile Feedback Co-Located With Virtual Touchscreen Improves Dual-Task Performance

The use of haptic technology has recently become essential in Human-Computer Interaction to improve performance and user experience. Mid-air tactile feedback co-located with virtual touchscreen displays have a great potential to improve the performance in dual-task situations, such as when using a phone while walking or driving. The purpose of this article is to investigate the effects of augmenting virtual touchscreen with mid-air tactile feedback to improve dual-task performance where the primary task is driving in a simulation environment and the secondary task involves interacting with a virtual touchscreen. Performance metrics included primary task performance in terms of velocity error, deviation from the middle of the road, number of collisions, and the number of off-road glances, secondary task performance including the interaction time and the reach time, and quality of user experience for perceived difficulty and satisfaction. Results demonstrate that adding mid-air tactile feedback to virtual touchscreen resulted in statistically significant improvement in the primary task performance (the average speed error, spatial deviation, and the number of off-road glances), the secondary task (reach time), and the perceived difficulty. These results provide a great motivation for augmenting virtual touchscreens with mid-air tactile feedback in dual-task human-computer interaction applications.

The Evaluation of Tactile Parameters and Display Prototype to Support Physiological Monitoring and Multitasking for Anesthesia Providers in the Operating Room

Communicating physiological information via the tactile modality is shown as a promising means to address data overload faced by anesthesia providers. However, it is important to ensure that the tactile parameters which represent information are intuitive. There is currently no consensus on which tactile parameters should be used to present information within anesthesia. The two studies presented here evaluate: (a) a set of 24 tactile cues manipulating intensity, temporal, and spatial tactile parameters in a usability study and (b) a prototype tactile display based on the usability study's findings in a single and dual-task scenario. Findings of the usability study show intensity and temporal were rated most urgent and had the most potential to represent changes in physiological measures. This was confirmed in the follow up study as increases/decreases in intensity were shown to represent increases/decreases in a physiological measure and using different spatial locations to represent physiological measures resulted in greater than 95% response accuracy. Response times and accuracy were not adversely affected while performing a secondary task. The findings contribute to a better understanding of how to map tactile parameters to physiological information and demonstrate the effectiveness of end-user feedback in tactile display design to develop intuitive alerts.

Now You Feel It, Now You Don't: The Effect of Movement, Cue Complexity, and Body Location on Tactile Change Detection

OBJECTIVE

To evaluate the effects that movement, cue complexity, and the location of tactile displays on the body have on tactile change detection.

BACKGROUND

Tactile displays have been demonstrated as a means to address data overload by offloading the visual and auditory modalities. However, change blindness-the failure to detect changes in a stimulus when changes coincide with another event or disruption in stimulus continuity-has been demonstrated to affect the tactile modality and may be exacerbated during movement. The complexity of tactile cues and locations of tactile displays on the body may also affect the detection of changes in tactile patterns. Limitations to tactile perception need to be examined.

METHOD

Twenty-four participants performed a tactile change detection task while sitting, standing, and walking. Tactile cues varied in complexity and included low, medium, and high complexity cues presented to the arm or back.

RESULTS

Movement adversely affects tactile change detection as hit rates were the highest while sitting, followed by standing and walking. Cue complexity affected tactile change detection: Low complexity cues resulted in higher detection rates compared with medium and high complexity cues. The arms exhibited better change detection performance than the back.

CONCLUSION

The design of tactile displays should consider the effect of movement. Cue complexity should be minimized and decisions about the location of a tactile display should take into account body movements to support tactile perception.

APPLICATION

The findings can provide design guidelines to inform tactile display design for data-rich, complex domains.

Component-Based Interactive Framework for Intelligent Transportation Cyber-Physical Systems

While emerging technology for self-driving automation in vehicles progresses rapidly, the transition to an era of roads full of fully connected and automated vehicles (CAVs) may take longer than expected. Until then, it is inevitable that CAVs should coexist and interact with drivers of non-autonomous vehicles (NAVs) in urban roads. During this period of transition, it is critical to provide road safety with the mixed vehicular traffic and uncertainty caused by human drivers. To investigate the issues caused by the coexistence and interaction with humans, we propose to build a component-based and interactive intelligent transportation cyber-physical systems (ITCPS) framework. Our design of the interactive ITCPS framework aims to provide a standardized structure for users by defining core components. The framework is specified by behavior models and interfaces for the desired ITCPS components and is implemented as a form of human and hardware-in-the-loop system. We developed an intersection crossing assistance service and an automatic emergency braking service as an example of practical applications using the framework. To evaluate the framework, we tested its performance to show how effectively it operates while supporting real-time processing. The results indicate that it satisfies the timing requirements of vehicle-to-vehicle (V2V) communication and the limited processing time required for performing the functions of behavior models, even though the traffic volume reaches the road capacity. A case study using statistical analysis is conducted to assess the practical value of the developed experimental environment. The results of the case study validate the reliability among the specified variables for the experiments involving human drivers. It has shown that V2V communication support has positive effects on road safety, including intersection safety, braking events, and perception-reaction time (PRT) of the drivers. Furthermore, V2V communication support and PRT are identified as the important indicators affecting road safety at an un-signalized intersection. The proposed interactive framework is expected to contribute in constructing a comprehensive environment for the urban ITCPS and providing experimental support for the analysis of human behavior in the coexistence environment.

Age, cognitive load, and multimodal effects on driver response to directional warning

Inattention can be considered a primary cause of vehicular accidents or crashes, and in-car warning signals are applied to alert the driver to take action even in automated vehicles. Because of age related decline of the older driver's abilities, in-car warning signals may need adjustment to the older driver. We therefore investigated the effects of uni-, bi- and trimodal directional warnings (i.e., light, sound, vibration) on young and older drivers' responses in a driving simulator. A young group of 15 drivers (20-25 years of age) and an older group of 16 drivers (65-79 years of age) participated. In the simulations, warning signal was presented at the left, the center, or the right in front of the participant. With a warning at the left, the center, and the right the correct response was to steer to the right, brake, and steer to the left, respectively. The main results showed the older drivers' responses were slower for each type of warning compared with the young drivers' responses. Overall, the responses were slower with an added cognitively loading task. The only multimodal type of warning inducing overall faster response than its constituent warning types was the vibration-sound, and only for the older drivers. Additionally, with the groups' responses collapsed, such a true multimodal effect on response time also showed for the center vibration-sound warning (i.e., braking response). The only multimodal warning showing clear reduction in response errors compared with its constituent warning types was the vibration-sound for the older drivers during extra cognitive load. The main conclusion is that older drivers can benefit from bimodal warning, as compared with unimodal, in terms of faster and more accurate response. The potential superiority of trimodal warning is nevertheless argued.

Tactile, Visual, and Crossmodal Visual-Tactile Change Blindness: The Effect of Transient Type and Task Demands

OBJECTIVE

The present study examined whether tactile change blindness and crossmodal visual-tactile change blindness occur in the presence of two transient types and whether their incidence is affected by the addition of a concurrent task.

BACKGROUND

Multimodal and tactile displays have been proposed as a promising means to overcome data overload and support attention management. To ensure the effectiveness of these displays, researchers must examine possible limitations of human information processing, such as tactile and crossmodal change blindness.

METHOD

Twenty participants performed a unmanned aerial vehicle (UAV) monitoring task that included visual and tactile cues. They completed four blocks of 70 trials each, one involving visual transients, the other tactile transients. A search task was added to determine whether increased workload leads to a higher risk of change blindness.

RESULTS

The findings confirm that tactile change detection suffers in terms of response accuracy, sensitivity, and response bias in the presence of a tactile transient. Crossmodal visual-tactile change blindness was not observed. Also, change detection was not affected by the addition of the search task and helped reduce response bias.

CONCLUSION

Tactile displays can help support multitasking and attention management, but their design needs to account for tactile change blindness. Simultaneous presentation of multiple tactile indications should be avoided as it adversely affects change detection.

APPLICATION

The findings from this research will help inform the design of multimodal and tactile interfaces in data-rich domains, such as military operations, aviation, and healthcare.

Effects of Workload and Auditory Interruptions on Emergency Dispatching Performance

Interruptions and workload can cause frustration and decrements in performance, which can be especially detrimental in complex domains such as emergency response. Moreover, interruptions and notifications in such domains may be presented across different modalities, such as vision and audition. In this study, we are specifically interested in evaluating the effects of workload and auditory interruptions on performance in emergency dispatching, a complex and dynamic environment in which an operator must make time- and safety-critical decisions based on information presented across different modalities. An experiment was conducted with 41 student participants using a simulated emergency dispatching task. Auditory interruptions simulating new emergencies were introduced within the main task, which is to monitor the status of different emergency aid vehicles. Different scenarios were tested where workload was varied between low and high. Results suggest degraded performance in both the primary and interruptive task in high workload. Results also suggest a better ability to process the visual primary task while auditory information was presented in low workload than in high workload. The results add to the literature on the effects of interruptions and workload in complex domains, and their implications for designing displays in such domains.

Effects of Real-Time (Sonification) and Rhythmic Auditory Stimuli on Recovering Arm Function Post Stroke: A Systematic Review and Meta-Analysis

Background: External auditory stimuli have been widely used for recovering arm function post-stroke. Rhythmic and real-time auditory stimuli have been reported to enhance motor recovery by facilitating perceptuomotor representation, cross-modal processing, and neural plasticity. However, a consensus as to their influence for recovering arm function post-stroke is still warranted because of high variability noted in research methods. Objective: A systematic review and meta-analysis was carried out to analyze the effects of rhythmic and real-time auditory stimuli on arm recovery post stroke. Method: Systematic identification of published literature was performed according to PRISMA guidelines, from inception until December 2017, on online databases: Web of science, PEDro, EBSCO, MEDLINE, Cochrane, EMBASE, and PROQUEST. Studies were critically appraised using PEDro scale. Results: Of 1,889 records, 23 studies which involved 585 (226 females/359 males) patients met our inclusion criteria. The meta-analysis revealed beneficial effects of training with both types of auditory inputs for Fugl-Meyer assessment (Hedge's g: 0.79), Stroke impact scale (0.95), elbow range of motion (0.37), and reduction in wolf motor function time test (-0.55). Upon further comparison, a beneficial effect of real-time auditory feedback was found over rhythmic auditory cueing for Fugl-meyer assessment (1.3 as compared to 0.6). Moreover, the findings suggest a training dosage of 30 min to 1 h for at least 3-5 sessions per week with either of the auditory stimuli. Conclusion: This review suggests the application of external auditory stimuli for recovering arm functioning post-stroke.

Auditory Proprioceptive Integration: Effects of Real-Time Kinematic Auditory Feedback on Knee Proprioception

The purpose of the study was to assess the influence of real-time auditory feedback on knee proprioception. Thirty healthy participants were randomly allocated to control (n = 15), and experimental group I (15). The participants performed an active knee-repositioning task using their dominant leg, with/without additional real-time auditory feedback where the frequency was mapped in a convergent manner to two different target angles (40 and 75°). Statistical analysis revealed significant enhancement in knee re-positioning accuracy for the constant and absolute error with real-time auditory feedback, within and across the groups. Besides this convergent condition, we established a second divergent condition. Here, a step-wise transposition of frequency was performed to explore whether a systematic tuning between auditory-proprioceptive repositioning exists. No significant effects were identified in this divergent auditory feedback condition. An additional experimental group II (n = 20) was further included. Here, we investigated the influence of a larger magnitude and directional change of step-wise transposition of the frequency. In a first step, results confirm the findings of experiment I. Moreover, significant effects on knee auditory-proprioception repositioning were evident when divergent auditory feedback was applied. During the step-wise transposition participants showed systematic modulation of knee movements in the opposite direction of transposition. We confirm that knee re-positioning accuracy can be enhanced with concurrent application of real-time auditory feedback and that knee re-positioning can modulated in a goal-directed manner with step-wise transposition of frequency. Clinical implications are discussed with respect to joint position sense in rehabilitation settings.

Enhancing the effectiveness of human-robot teaming with a closed-loop system

With technological developments in robotics and their increasing deployment, human-robot teams are set to be a mainstay in the future. To develop robots that possess teaming capabilities, such as being able to communicate implicitly, the present study implemented a closed-loop system. This system enabled the robot to provide adaptive aid without the need for explicit commands from the human teammate, through the use of multiple physiological workload measures. Such measures of workload vary in sensitivity and there is large inter-individual variability in physiological responses to imposed taskload. Workload models enacted via closed-loop system should accommodate such individual variability. The present research investigated the effects of the adaptive robot aid vs. imposed aid on performance and workload. Results showed that adaptive robot aid driven by an individualized workload model for physiological response resulted in greater improvements in performance compared to aid that was simply imposed by the system.

Effectiveness of auditory and tactile crossmodal cues in a dual-task visual and auditory scenario

In this study, we examined how spatially informative auditory and tactile cues affected participants' performance on a visual search task while they simultaneously performed a secondary auditory task. Visual search task performance was assessed via reaction time and accuracy. Tactile and auditory cues provided the approximate location of the visual target within the search display. The inclusion of tactile and auditory cues improved performance in comparison to the no-cue baseline conditions. In comparison to the no-cue conditions, both tactile and auditory cues resulted in faster response times in the visual search only (single task) and visual-auditory (dual-task) conditions. However, the effectiveness of auditory and tactile cueing for visual task accuracy was shown to be dependent on task-type condition. Crossmodal cueing remains a viable strategy for improving task performance without increasing attentional load within a singular sensory modality. Practitioner Summary: Crossmodal cueing with dual-task performance has not been widely explored, yet has practical applications. We examined the effects of auditory and tactile crossmodal cues on visual search performance, with and without a secondary auditory task. Tactile cues aided visual search accuracy when also engaged in a secondary auditory task, whereas auditory cues did not.

Effect of Redundant Haptic Information on Task Performance during Visuo-Tactile Task Interruption and Recovery

Previous studies have revealed that interruption induces disruptive influences on the performance of cognitive tasks. While much research has focused on the use of multimodal channels to reduce the cost of interruption, few studies have utilized haptic information as more than an associative cue. In the present study, we utilized a multimodal task interruption scenario involving the simultaneous presentation of visual information and haptic stimuli in order to investigate how the combined stimuli affect performance on the primary task (cost of interruption). Participants were asked to perform a two-back visuo-tactile task, in which visual and haptic stimuli were presented simultaneously, which was interrupted by a secondary task that also utilized visual and haptic stimuli. Four experimental conditions were evaluated: (1) paired information (visual stimulus + paired haptic stimulus) with interruption; (2) paired information without interruption; (3) non-paired information (visual stimulus + non-paired haptic stimulus) with interruption; and (4) non-paired information without interruption. Our findings indicate that, within a visuo-tactile task environment, redundant haptic information may not only increase accuracy on the primary task but also reduce the cost of interruption in terms of accuracy. These results suggest a new way of understanding the task recovery process within a multimodal environment.

Time Sharing Between Robotics and Process Control: Validating a Model of Attention Switching

OBJECTIVE

The aim of this study was to validate the strategic task overload management (STOM) model that predicts task switching when concurrence is impossible.

BACKGROUND

The STOM model predicts that in overload, tasks will be switched to, to the extent that they are attractive on task attributes of high priority, interest, and salience and low difficulty. But more-difficult tasks are less likely to be switched away from once they are being performed.

METHOD

In Experiment 1, participants performed four tasks of the Multi-Attribute Task Battery and provided task-switching data to inform the role of difficulty and priority. In Experiment 2, participants concurrently performed an environmental control task and a robotic arm simulation. Workload was varied by automation of arm movement and both the phases of environmental control and existence of decision support for fault management. Attention to the two tasks was measured using a head tracker.

RESULTS

Experiment 1 revealed the lack of influence of task priority and confirmed the differing roles of task difficulty. In Experiment 2, the percentage attention allocation across the eight conditions was predicted by the STOM model when participants rated the four attributes. Model predictions were compared against empirical data and accounted for over 95% of variance in task allocation. More-difficult tasks were performed longer than easier tasks. Task priority does not influence allocation.

CONCLUSIONS

The multiattribute decision model provided a good fit to the data.

APPLICATIONS

The STOM model is useful for predicting cognitive tunneling given that human-in-the-loop simulation is time-consuming and expensive.

Development and experimental evaluation of an alarm concept for an integrated surgical workstation

Alarm conditions of the technical equipment in operating rooms represent a prevalent cause for interruptions of surgeons and scrub nurses, resulting in an increase of workload and potential reduction of patient safety. In this work, an alarm concept for an integrated operating room system based on open communication standards is developed and tested.

In a laboratory experiment, the reactions of surgeons were analysed, comparing the displaying of alarms on an integrated workstation and on single devices: disruptive effects of alarm handling on primary task (ratings of perceived distraction, resumption lag, deterioration of speed, accuracy, and prospective memory), efficiency and effectiveness of identification of alarms, as well as perceived workload were included.

The identification of the alarm cause is significantly more efficient and effective with the integrated alarm concept. Moreover, a slightly lower deterioration of performance of the primary task due to the interruption of alarm handling was observed.

Displaying alarms on an integrated workstation supports alarm handling and consequently reduces disruptive effects on the primary task. The findings show that even small changes can reduce workload in a complex work environment like the operating room, resulting in improved patient safety.

Effects of inter-stimulus interval and intensity on the perceived urgency of tactile patterns

This research examines the feasibility of coding urgency into tactile patterns. Four tactile patterns were presented at either, 12 or 23.5 dB above mean threshold, with an ISI of either 0 (no interval) or 500 msec. Measures included pattern identification and urgency rating on a scale of 1 (least urgent) to 10 (most urgent). Two studies were conducted, a laboratory study and a field study. In the laboratory study, participants received the tactile patterns while seated in front of a computer. For the field study, participants performed dismounted Soldier maneuvers while receiving the tactile patterns. Higher identification rates were found for the 23.5 dB intensity. Patterns presented at the 23.5 dB intensity and no ISI were rated most urgent. No differences in urgency ratings were found for 12 dB based on ISI. Findings support the notion of coding urgency into tactile patterns as a way of augmenting tactile communication.

Tracking and discrete dual task performance with different spatial stimulus-response mappings

The effect of spatial compatibility on dual-task performance for various display-control configurations was studied using a tracking task and a discrete four-choice response task. A total of 36 participants took part in this study, and they were asked to perform the primary tracking task while at the same time to respond to an occasional signal. Different levels of compatibility between the stimuli and responses of the discrete response task were found to lead to different degrees of influence on the tracking task. However, degradation of performance was observed for both tasks, which was probably due to resource competition for the visual and spatial resources required for simultaneous task operation and required for bimanual responses. No right-left prevalence effect for the spatial compatibility task was observed in this study, implying that the use of unimanual two-finger responses may not provide the right conditions for a significant effect in the horizontal right-left dimension.

PRACTITIONER SUMMARY

The effect of spatial compatibility in multiple display-control configurations was examined in a dual-task paradigm. The analyses of keen competition for visual and spatial resources in processing the dual tasks under different degrees of stimulus-response compatibility provide useful ergonomics design implications and recommendations for visual interfaces requiring frequent visual scanning.

Workload overload modeling

Task switching choice was examined building from a model of task overload management. An experiment using the Multi-Attribute Task Battery (MATB) was undertaken to explore the influence of two parameters of the model, task priority and task difficulty. Participants were free to switch between the four component tasks, with the number of switches and task choice for conflicting events observed. A unique post-experiment survey measured subjective ratings of task attributes. We found that task difficulty, by reducing switching, and task priority, which determined whether increased task difficulty increased time in task, significantly influenced task switching predominantly in line with our predictions. The specific role of priority in multi-task management, and future directions including time-on-task related effects and the role of operator fatigue, are discussed.

The Integrative Team Knowledge Building Training Strategy in Distributed Problem-Solving Teams

Team knowledge building requires teams to discuss all available pertinent task information. However, teams tend to extract information ineffectively due to impediments including the sampling advantage of common information and differential schema structures among team members. In addition, distributed teams have difficulty building knowledge due to constraints associated with low-bandwidth computer-mediated communication. Therefore, we tested a team training strategy aimed at facilitating team knowledge building in distributed teams. Data were collected from 40 teams of three distributed members. Teams were assigned to a training or control condition, and they completed a realistic problem-solving task. Training condition teams were trained to build knowledge by creating a knowledge object integrated with schema-enriched communication behaviors in text chat. Control condition teams communicated using chat only and received no training. Results indicated trained teams, relative to untrained teams, shared more unique information, transferred more knowledge, developed higher cognitive congruence, and produced higher quality solutions.

A Computational Model of Task Overload Management and Task Switching

We describe a computational model that predicts the decision aspect of sequential multitasking. We investigate how people choose to switch tasks or continue performing an ongoing task when they are in overload conditions where concurrent performance of tasks is impossible. The model is based on a metaanalytic integration of 46 experiments from two literatures: interruption management and applied task switching. Consistent trends from the meta-analysis are used to set parameters in the mathematical model, which is then implemented in a task network model.