Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology

The effect of information integration on team communication in a simulated submarine control room task

Submarine control rooms are characterised by dedicated individual roles for information types (e.g. Sonar operator processes sound energy), with individuals verbally reporting the information that they receive to other team members to help resolve uncertainty in the operational environment (low information integration). We compared this work design with one that ensured critical information was more readily available to all team members (high information integration). We used the Event Analysis of Systemic Teamwork (EAST) method to analyse task, information, and social networks for novice teams operating within a simulated submarine control room under low versus high information integration. Integration impacted team member centrality (importance relative to other operators) and the nature of information shared. Team members with greater centrality reported higher workload. Higher integration across consoles altered how team members interacted and their relative status, the information shared, and how workload was distributed. However, overall network structures remained intact.

When tomorrow comes: A prospective risk assessment of a future artificial general intelligence-based uncrewed combat aerial vehicle system

There are concerns that Artificial General Intelligence (AGI) could pose an existential threat to humanity; however, as AGI does not yet exist it is difficult to prospectively identify risks and develop requisite controls. We applied the Work Domain Analysis Broken Nodes (WDA-BN) and Event Analysis of Systemic Teamwork-Broken Links (EAST-BL) methods to identify potential risks in a future 'envisioned world' AGI-based uncrewed combat aerial vehicle system. The findings suggest five main categories of risk in this context: sub-optimal performance risks, goal alignment risks, super-intelligence risks, over-control risks, and enfeeblement risks. Two of these categories, goal alignment risks and super-intelligence risks, have not previously been encountered or dealt with in conventional safety management systems. Whereas most of the identified sub-optimal performance risks can be managed through existing defence design lifecycle processes, we propose that work is required to develop controls to manage the other risks identified. These include controls on AGI developers, controls within the AGI itself, and broader sociotechnical system controls.

Human-centred explanations for artificial intelligence systems

As Artificial Intelligence (AI) systems increase in capability, so there are growing concerns over the ways in which the recommendations they provide can affect people's everyday life and decisions. The field of Explainable AI (XAI) aims to address such concerns but there is often a neglect of the human in this process. We present a formal definition of human-centred XAI and illustrate the application of this formalism to the design of a user interface. The user interface supports users in indicating their preferences relevant to a situation and to compare their preferences with those of a computer recommendation system. A user trial is conducted to evaluate the resulting user interface. From the user trial, we believe that users are able to appreciate how their preferences can influence computer recommendations, and how these might contrast with the preferences used by the computer. We provide guidelines of implementing human-centred XAI.

Building Bridges Instead of Putting Up Walls: Connecting the "Teams" to Improve Soccer Players' Support

The increase in the economic value of soccer occurred in parallel with an increase in competing demands. Therefore, clubs and federations evolved to greater specialization (e.g., state-of-the-art facilities and high-profile expertise staff) to support players' performance and health. Currently, player preparation is far from exclusively club or national team centered, and the lack of control in each player's environment can be more prevalent than expected. For example, an elite group of professional players faces disruptions in the season club-oriented planification due to involvement in national teams. Moreover, as elite players' financial resources grow, it is common for them to employ specialized personal staff (e.g., strength and conditioning, nutritionist, and sports psychologist) to assist in their preparation, resulting in complex three-fold relationships (i.e., club, player's staff, national team). Although efforts have been made to improve communication with and transition from the club to the national team supervision, this new reality (club-players' staff) may generate serious compound role-related problems and difficulties in monitoring load and training adaptation and having a unified message. Therefore, efforts must be implemented to ensure a more informed management of the players' performance environment, where the existence and impact of these various personal staff are considered to avoid a long-term non-zero sum for all intervening parties. If left unchecked, current professional thinking may collide or overlap, potentially triggering conflict escalation and impairing athletic performance or health, especially if effective communication routes are not adequately established. Moreover, diluted personal responsibility regarding performance may ensue, resulting in decreased productivity from all involved, which may cause more harm than benefits for the player's overall health and performance. This emerging reality calls for developing a joint working framework (i.e., between the player's personalized support team and the clubs' team) and better managing of a player-centered process.

Forecasting emergent risks in advanced AI systems: an analysis of a future road transport management system

Artificial Intelligence (AI) is being increasingly implemented within road transport systems worldwide. Next generation of AI, Artificial General Intelligence (AGI) is imminent, and is anticipated to be more powerful than current AI. AGI systems will have a broad range of abilities and be able to perform multiple cognitive tasks akin to humans that will likely produce many expected benefits, but also potential risks. This study applied the EAST Broken Links approach to forecast the functioning of an AGI system tasked with managing a road transport system and identify potential risks. In total, 363 risks were identified that could have adverse impacts on the stated goals of safety, efficiency, environmental sustainability, and economic performance of the road system. Further, risks beyond the stated goals were identified; removal from human control, mismanaging public relations, and self-preservation. A diverse set of systemic controls will be required when designing, implementing, and operating future advanced technologies.Practitioner summary: This study demonstrated the utility of HFE methods for formally considering risks associated with the design, implementation, and operation of future technologies. This study has implications for AGI research, design, and development to ensure safe and ethical AGI implementation.

The tendency of the schematic structure to maintain stability can be interpreted as mental inertia

This paper incorporates schematic concepts related to mental inertia and provides an avenue for interpreting psychology using the principles of classical mechanics. Schemas find wide application in diverse fields, ranging from ergonomics to psychotherapy. Nonetheless, it is crucial to incorporate schemas themselves into a more unified and comprehensive theoretical framework. Drawing upon the free energy principle (FEP) and the second law of thermodynamics, it is evident that humans possess a natural inclination to construct and maintain consistent cognitive structures. This characteristic contributes to the stability of schemas within a defined range. The particular scope of the model is closely intertwined with its structure, leading to variations among individuals in diverse environments. The coherence of the schema within a defined range can be perceived as the magnitude of mental inertia. This psychological analogy emphasizes the importance of considering the influences exerted by the external environment and their effects on mental inertia when predicting the human mind and behavior.

Leader airtime management and team effectiveness in emergency management command and control (EMCC) teams

We investigated the relation between leader airtime management and team effectiveness in Emergency Management Command and Control (EMCC) teams. Leader airtime management concerns leaders' interventions to structure who shares information when using opening and closing statements to respectively stimulate or reduce information sharing. We coded leaders' airtime management statements across different meeting phases (structuring, information sharing, decision making) using video-recordings of 12 EMCC exercises involving two consecutive meetings each. Experts rated two components of team effectiveness: Team Situation Awareness (TSA) and Team Decision Making (TDM). We found that closing statements were more frequently used in the decision-making phase than in any other meeting phase. Also, leaders of teams with lower TSA used more opening statements in the decision-making phase of the first team meeting than leaders of teams with higher TSA. These results confirm the importance of the timing of leader airtime management for EMCC team effectiveness. Practitioner summary: We investigated leader airtime management and team effectiveness in EMCC teams. We video-coded 12 exercises; experts rated team effectiveness. In the decision-making phase, leaders use more closing statements, and leaders of less effective teams use more opening statements. Leaders are advised to adjust their airtime management to meeting phases.

Analysing dynamic work systems using DynEAST: a demonstration of concept

The capability of current Ergonomics methods to capture dynamism is limited, stifling our understanding of work-as-done, distributed situational awareness and organisational drift. This paper provides a demonstration of concept of DynEAST; an extension of the EAST framework underpinned by principles from Dynamic Network Analysis, to capture elements of dynamism within work systems. The DynEAST concept is applied to a railway maintenance case study. Case study findings demonstrate how DynEAST outputs can be used to advance our understanding of the aforementioned phenomena and better equip practitioners for current and future Ergonomics challenges.Practitioner summary: This paper introduces the DynEAST method. DynEAST enables HF/E practitioners to model and analyse dynamic features of complex work systems. The development of DynEAST is timely due to the concurrent proliferation of increasingly complex sociotechnical systems and stagnation of HF/E methods development; particularly those able to model systemic dynamism. Abbreviations: DynEAST: dynamic event analysis of systemic teamwork; EAST: dynamic event analysis of systemic teamwork; HF/E: human factors and ergonomics; HF: human factors; DNA: dynamic network analysis; HTA: hierarchal task analysis; CWA: cognitive work analysis; CAST: causal analysis based on system theory; STAMP: system theoretic accident model and processes; FRAM: functional resonance analysis method; SNA: social network analysis; DSA: distributed situational awareness; PPO: possession protection officer; PO: protection officer; RTS: railway track signals; LPA: local possession authority; SMEs: subject matter experts.

A novel study of situational awareness among out-of-hospital providers during an online clinical simulation

BACKGROUND

Out-of-hospital providers must be situationally aware in order to maintain safety for themselves, their patients, the public, and their crew. They must be able to recognize situations, interpret them, and predict how those situations may unfold in the future.

METHOD

A mixed-methods explanatory sequential design where out-of-hospital providers in Oklahoma, USA, participated in an 18-minute online simulation and then had their situational awareness (SA) measured. Upon completion, participants provided feedback during scheduled interviews.

RESULTS

A total of 156 out-of-hospital providers participated. Participants were not situationally aware. While not statistically significant, those with higher education had higher SA scores. Participants perceived that the simulation processes were beneficial to their ongoing care in the field, and were satisfied with the online simulation environment.

CONCLUSIONS

Out-of-hospital providers are not situationally aware during an online clinical simulation. While they are focused intently on the surroundings of the scene, they do not put that same focus on the patient. With the appropriate technology and setup, the use of the Situational Awareness Global Assessment Technique (SAGAT) during online simulation is feasible and could enhance clinical performance. Further studies are needed to determine if overall education or years of clinical experience play a role in out-of-hospital SA.

“Dare to Share”: Improving Information Sharing and Risk Assessment in Multiteam Systems Managing Offender Probation

Managing offenders released from prison requires a multiteam system (MTS) approach, with teams from across police, probation, and other criminal justice agencies. However, public inquiries highlight current approaches are impaired by poor information sharing that compromises risk assessment and recall decisions, which can allow serious further offences to occur. Little research has focused on the causes of these information sharing difficulties. The current study draws on the perspectives and experiences of probation and police officers to improve understanding of causes of information sharing difficulties. The research is conducted within the context of a new enhanced information sharing ‘Direct Access’ initiative implemented in one region of the UK (Devon and Cornwall) that permits probation to directly access police information technology systems. This provides a novel opportunity to cross validate MTS theory to the real-world context of offender management and considers what works in practise to overcome information sharing challenges. Eleven semi-structured interviews were conducted with police (N = 4) and probation (N = 7) officers. Thematic analysis revealed six themes: i) information sharing difficulties and impact; ii) causes of information sharing difficulties; iii) impact of ‘Direct Access’ on information sharing practices; iv) workload inequality; v) training; and vi) evolution of ‘Direct Access’. Overall, findings highlight that information sharing difficulties are causes by not knowing what information to request or share, limited resources, lack of clarity about General Data Protection Regulation and concern about consequences of breaching this. These barriers can result in delays and failures to share information, which hinders the accuracy of risk assessments and ability to safeguard. Findings also highlight that providing statutory partners with ‘Direct Access’ to I.T systems can improve the relevance and timeliness of information. However, ‘daring to share’ is not enough to address trust issues without also clarifying expectations regarding information use and perceived workload inequalities.

Can a targeted educational approach improve situational awareness in paramedicine during 911 emergency calls?

BACKGROUND

Paramedics must be situationally aware in order to avoid human error and protect themselves, their partner, their patient, and the public. Previous research has suggested that paramedics lack situational awareness (SA), possibly due to a lack of an organized approach, distraction, and a poor understanding of SA. There is no educational approach provided to paramedics that is known to improve their levels of SA. If such an approach were provided, it could possibly reduce human error and lead to improved outcomes.

OBJECTIVE

The aim of this study is to determine whether providing paramedics with a targeted educational approach, including aspects from crew resource management (CRM) such as sterile cockpit and Endsley's model for SA, can improve overall SA during emergency calls.

METHODS

A prospective, quasi-experimental before-and-after study was used, in which out-of hospital paramedics were observed during 911 emergency calls. Baseline SA was measured using the situational awareness global assessment technique (SAGAT). Paramedics were then given a targeted educational lesson focusing on elements of SA and CRM as well as a novel quick reference tool to utilize during 911 calls. Post intervention SA was then measured again, and before-and-after results were compared for difference of mean scores.

RESULTS

The overall baseline SA was 62% increasing to 86% following the educational intervention. The Wilcoxon Ranked Sum was used to assess for statistically significant differences between mean SA performance in the same group of paramedics before and after the intervention. The increase in SA was found to be statistically significant where p = 0.011.

CONCLUSIONS

A targeted educational approach focusing on CRM and a novel quick reference tool may increase SA levels of paramedics during 911 emergency calls. Further studies are needed with bigger cohorts. Paramedicine educational institutions and out-of-hospital agencies should consider implementing this targeted approach with their students and currently practicing providers.

Information Requirements of a Decision Support System for Severe Accident Management in Nuclear Power Plants

In nuclear power plants, a severe accident is a critical accident involving significant nuclear core damage and it is managed by using a set of Severe Accident Management Guidelines (SAMG). Prepared as a guideline that provides lists of suggestions rather than strict instructions, SAMG’s contents require frequent decision-making by the operators, causing high cognitive load and creating an error-prone situation that is also amplified by the stressful environment during the severe accident mitigation efforts. A decision support system (DSS), designed by considering the human decision-making process and the system’s holistic view, can help the operators in making informed and appropriate decisions. In this study, we aim to identify the information requirements in designing such DSS for severe accident management of nuclear power plants. We combined two methods: Functional Resonance Analysis Method (FRAM) and decision ladder to identify the information requirements. FRAM provides a systematic analysis of the functions involved in severe accident management and decision ladder captures the human decision-making processes. We developed the FRAM model and the decision ladder model based on SAMG’s contents to identify the set of information requirements. The identified information requirements and their implementation suggestions are provided. This study is the first step in designing a decision support system that considers human cognitive load and holistic system concepts. The method used in this study shall contribute to the design and implementation of a DSS capable of supporting the operators in achieving safer decision-making, not only in nuclear power plants’ severe accident management but also in similar safety-critical systems.

State of science: models and methods for understanding and enhancing teams and teamwork in complex sociotechnical systems

This state of the science review brings together the disparate literature of effective strategies for enhancing and accelerating team performance. The review evaluates and synthesises models and proposes recommended avenues for future research. The two major models of the Input-Mediator-Output-Input (IMOI) framework and the Big Five dimensions of teamwork were reviewed and both will need significant development for application to future teams comprising non-human agents. Research suggests that a multi-method approach is appropriate for team measurements, such as the integration of methods from self-report, observer ratings, event-based measurement and automated recordings. Simulations are recommended as the most effective team-based training interventions. The impact of new technology and autonomous agents is discussed with respect to the changing nature of teamwork. In particular, whether existing teamwork models and measures are suitable to support the design, operation and evaluation of human-nonhuman teams of the future. Practitioner summary: This review recommends a multi-method approach to the measurement and evaluation of teamwork. Team models will need to be adapted to describe interaction with non-human agents, which is what the future is most likely to hold. The most effective team training interventions use simulation-based approaches.

Distributed situation awareness: From awareness in individuals and teams to the awareness of technologies, sociotechnical systems, and societies

A large component of Neville Stanton's work has focused on situation awareness in domains such as defence, transport, and process control. A significant contribution has been to initiate a shift from considering individual human operator situation awareness to considering the situation awareness of human and non-human teams, organisations, and even sociotechnical systems. Though controversial when introduced, the distributed situation awareness model has become increasingly relevant for modern day systems and problems. In this article we reflect on Stanton's contribution and point to a pressing need to consider a. The situation awareness of advanced technologies, and b. situation awareness at a sociotechnical system, societal and even global level. This is demonstrated via discussion on two contemporaneous issues: automated vehicles and the COVID-19 pandemic. It is concluded that, given advances such as artificial intelligence, the increased connectedness of society, emerging issues such as disinformation, and an increasing set of global threats, Stanton's distributed situation awareness model and associated analysis framework provide a useful toolkit for future Human Factors and Ergonomics applications.

Situational Awareness: Forecasting Successful Military Medical Teams

INTRODUCTION

Military healthcare providers working in military interprofessional healthcare teams (MIHTs) require situational awareness (SA) to ensure safe and efficacious patient care. This study aimed to explore SA in MIHTs to understand how SA can be reinforced and maintained in MIHTs. The research team set out to answer two questions: "What aspects of individual and team SA are particularly important for MIHTs?" and "How can we enable military healthcare providers to be effective MIHTs members with robust SA?".

METHODS

This study used Grounded Theory methodology collecting perspectives from 30 study participants from various backgrounds, including 11 different healthcare professions from the U.S. Army, Air Force, and Navy. Each study participant had experiences participating in, leading one, or leading many MIHTs. Data were collected in three cycles and analyzed within each cycle until saturation was reached.

RESULTS

Five themes were robustly represented in the data set regarding SA: (1) contextually informed adaptability, (2) readiness, (3) trust, (4) communication, and (5) mission focus.

CONCLUSIONS

The urgency often faced by MIHTs brings SA and the principles that underpin SA into sharper focus. The SA themes identified in this research may provide insight into training effectiveness, team strengths and weaknesses, and team performance.

Situation awareness and the mitigation of risk associated with patient deterioration: A meta-narrative review of theories and models and their relevance to nursing practice

BACKGROUND

Accurate situation awareness has been identified as a critical component of effective deteriorating patient response systems and an essential patient safety skill for nursing practice. However, situation awareness has been defined and theorised from multiple perspectives to explain how individuals, teams and systems maintain awareness in dynamic task environments.

AIM

Our aim was to critically analyse the different approaches taken to the study of situation awareness in healthcare and explore the implications for nursing practice and research as it relates to clinical deterioration in ward contexts.

METHODS

We undertook a meta-narrative review of the healthcare literature to capture how situation awareness has been defined, theorised and studied in healthcare. Following an initial scoping review, we conducted an extensive search of ten electronic databases and included any theoretical, empirical or critical papers with a primary focus on situation awareness in an inpatient hospital setting. Included papers were collaboratively categorised in accordance with their theoretical framing, research tradition and paradigm with a narrative review presented.

RESULTS

A total of 120 papers were included in this review. Three overarching narratives reflecting philosophical, patient safety and solution focussed framings of situation awareness and seven meta-narratives were identified as follows: individual, team and systems perspectives of situation awareness (meta-narratives 1-3), situation awareness and patient safety (meta-narrative 4), communication tools, technologies and education to support situation awareness (meta-narratives 5-7). We identified a concentration of literature from anaesthesia and operating rooms and a body of research largely located within a cognitive engineering tradition and a positivist research paradigm. Endsley's situation awareness model was applied in over 80% of the papers reviewed. A minority of papers drew on alternative situation awareness theories including constructivist, collaborative and distributed perspectives.

CONCLUSIONS

Nurses have a critical role in identifying and escalating the care of deteriorating patients. There is a need to build on prior studies and reflect on the reality of nurse's work and the constraints imposed on situation awareness by the demands of busy inpatient wards. We suggest that this will require an analysis that complements but goes beyond the dominant cognitive engineering tradition to reflect the complex socio-cultural reality of ward-based teams and to explore how situation awareness emerges in increasingly complex, technologically enabled distributed healthcare systems.

Team situation awareness accuracy measurement technique for simulated air combat - Curvilinear relationship between awareness and performance

A new technique for the assessment of Team Situation Awareness (TSA) accuracy based upon post task Critical Decision Method structured interviews was developed and tested using 39 combat-ready F/A-18 pilots. Pilots undertook a number of simulated air combat scenarios, flying in flights of four aircraft against a formation of enemy aircraft. Results showed a strong curvilinear relationship where high TSA accuracy resulted in higher performance in some areas of air combat, measured with friendly losses and kills. There were diminishing returns in performance as TSA accuracy increased. This may explain why previous studies on air combat have found relatively weak relationships between situation awareness and performance where the relationship has been assumed to be linear.

Do paramedic students have situational awareness during high-fidelity simulation? A mixed-methods pilot study

BACKGROUND

In order for paramedics to maintain safety for themselves, their partners, their patients, and the public, they must be situationally aware. To be situationally aware they must be able to recognize a situation, correctly interpret it, and analyse that information in order to anticipate future events. There are limited empirical data that identify whether or not paramedic students are situationally aware.

OBJECTIVE

The aim of this study is to determine if paramedic students are situationally aware during high-fidelity simulation.

METHOD

This mixed-methods pilot study was undertaken in a high-fidelity paramedic simulation lab. Twelve students participated in an out-of-hospital simulation, each while wearing a point-of-view camera. Data were derived from pre/post assessment surveys, performance assessments, debriefing with the situational awareness global assessment technique (SAGAT), and a full video debriefing. These data were used to determine whether students were able to recognize information, properly interpret that information, and accurately anticipate how that information would affect future events. Thematic analysis of the video debriefings was undertaken with emergent themes being extracted.

RESULTS

Quantitative data measures were derived from SAGAT data and performance assessments. Students recognized 42% (30/71) of the physiological and global items related to an emergency patient and the emergency scene itself. Of the items they recognized, they properly interpreted 34% (15/44) of them, leading to the ability to properly project 40% (24/60) of patient clinical progression. During student interviews, several themes were identified as key factors contributing to their loss of situational awareness, inlcuding tunnel vision, stress, and lack of an organized approach.

CONCLUSIONS

This study suggests that paramedic students are not situationally aware. Students failed to recognize some events. Of the events they did recognize, they were unable to properly interpret what they meant or how future events would unfold. It appears students lack situational awareness due to being stressed, failing to focus, and lacking an organized approach. Further research is needed to determine improved best practices in paramedic situational awareness education.

Collaborating AI and human experts in the maintenance domain

Maintenance decision errors can result in very costly problems. The 4th industrial revolution has given new opportunities for the development of and use of intelligent decision support systems. With these technological advancements, key concerns focus on gaining a better understanding of the linkage between the technicians’ knowledge and the intelligent decision support systems. The research reported in this study has two primary objectives. (1) To propose a theoretical model that links technicians’ knowledge and intelligent decision support systems, and (2) to present a use case how to apply the theoretical model. The foundation of the new model builds upon two main streams of study in the decision support literature: “distribution” of knowledge among different agents, and “collaboration” of knowledge for reaching a shared goal. This study resulted in the identification of two main gaps: firstly, there must be a greater focus upon the technicians’ knowledge; secondly, technicians need assistance to maintain their focus on the big picture. We used the cognitive fit theory, and the theory of distributed situation awareness to propose the new theoretical model called “distributed collaborative awareness model.” The model considers both explicit and implicit knowledge and accommodates the dynamic challenges involved in operational level maintenance. As an application of this model, we identify and recommend some technological developments required in augmented reality based maintenance decision support.

Cognition in crisis management teams: an integrative analysis of definitions

In large-scale extreme events, multidisciplinary crisis management teams (CMTs) are required to function together cognitively. Despite theoretical maturity in team cognition and recurrent emphasis on cognition in the crisis management practices, no synthesis of theoretical and practical discourses is currently available, limiting empirical investigations of cognition in CMTs. To address this gap, this paper aims to review the definitions of cognition in CMTs, with a particular focus on examining if and to what extent they are diversified. Through a systematic process to search peer-reviewed journal articles published in English from 1990 to 2019, 59 articles were selected with 62 coded definitions of 11 different constructs. The similarities and variabilities of the definitions were examined in terms of their theoretical and practical emphases and then synthesised into an integrative definition expected to serve as a general guide of reference for future researchers seeking an operational definition of cognition in CMTs. Practitioner summary: Understanding of cognition in CMTs is grounded in various theories and models with varying assumptions. An integrative conceptualisation of such cognition as interaction within and across CMTs to perceive, diagnose, and adapt to the crisis may facilitate the accumulation of knowledge and future operationalisations. Abbreviations: CMT(s): crisis management team(s); SMM: shared mental model; TMM: team mental model; COP: common operating picture; SSA: shared situation awareness; TSA: team situation awareness; DC: distributed cognition; ITC: interactive team cognition; TMS: transitive memory system(s); DSA: distributed situation awareness.

Distributed situation awareness: a health-system approach to assessing and designing patient flow management

Patient flow management is a system-wide process but many healthcare providers do not integrate multiple departments into the process to minimise the time between treatments or medical services for maximum patient throughput. This paper presents a case study of applying Distributed Situation Awareness (DSA) to characterise system-wide patient flow management and identify opportunities for improvements in a healthcare system. This case study employed a three-part method of data elicitation, extraction, and representation to investigate DSA. Social, task, and knowledge networks were developed and then combined to characterise patient flow management and identify deficiencies of the command and control centre of a healthcare facility. Social network analysis provided centrality metrics to further characterise patient flow management. The DSA model helped identify design principles and deficiencies in managing patient flow. These findings indicate that DSA is promising for analysing patient flow management from a system-wide perspective. Practitioner summary: This article examines Distribution Situation Awareness (DSA) as an analysis framework to study system-wide patient flow management. The DSA yields social, task, and knowledge networks that can be combined to characterise patient flow and identify deficiencies in the system. DSA appears promising for analysing communication and coordination of complex systems. Abbreviations: CDM: critical decision method; CTaC: carilion transfer and communications center; EAST: event analysis systematic teamwork; ED: emergency department; DES: discrete event simulation; DSA: distributed situation awareness; SA: situation awareness; SNA: social network analysis.

Conceptualizing caregiving activities for persons with dementia (PwD) through a patient work lens

Informal caregivers are an integral part of care delivery for persons with dementia (PwD). Informal caregivers take part in a wide range of care activities both individually and collaboratively with other caregivers. Caregiving often involves high demands in the face of limited resources, which can lead to stress, burden, and burnout. To support caregivers, we need to conceptualize caregiving activities they perform, and the networks and roles through which they perform work. We performed a directed content analysis on interview data from twenty caregivers and applied a human factors approach to characterize informal caregiving work. Our results revealed 1) nuances in caregiving roles, 2) differences in caregiving networks, and 3) 13 categories of caregiving activities characterized by time commitments; physical, cognitive and socio-behavioral demands; and varying network dependencies. These findings can be applied in future studies to evaluate the needs of caregiving networks and how to better support them.

“Are You Planning to Follow Your Route?” The Effect of Route Exchange on Decision Making, Trust, and Safety

The Sea Traffic Management (STM) Validation project is a European based initiative which focuses on connecting and updating the maritime world in real time, with efficient information exchange. The purpose of this paper is to evaluate two functions developed during the project: a ship to ship route exchange (S2SREX) function and rendezvous (RDV) information layer, collectively referred to as S2SREX/RDV. S2SREX displays the route segment consisting of the next seven waypoints of the monitored route of a collaborating ship and the RDV layer that predicts a meeting point. S2SREX/RDV provides supplementary information to data acquired by existing navigation systems and is intended to improve situational awareness and safety through a more comprehensive understanding of the surrounding traffic. Chalmers University of Technology and Solent University completed an experiment using twenty-four experienced navigators in bridge simulators. Six traffic scenarios were developed by subject matter experts and tested with and without S2SREX/RDV functionalities. Qualitative data were collected using post-test questionnaires and group debriefs to evaluate the participants’ perceptions of S2SREX/RDV in the various traffic scenarios, and quantitative data were collected to assess the ship distances and behavior in relation to the International Regulations for Preventing Collisions at Sea (COLREGs). The results revealed that participants generally trusted the S2SREX/RDV information, and most used S2SREX/RDV for decision support. The quantitative assessment revealed that the COLREGs were breached more often when S2SREX/RDV was used. Experimental findings are discussed in relation to safety, trust, reliance, situational awareness, and human-automation interaction constructs.

Improving Team-Based Decision Making Using Data Analytics and Informatics: Protocol for a Collaborative Decision Support Design

Background

According to the September 2015 Institute of Medicine report, Improving Diagnosis in Health Care, each of us is likely to experience one diagnostic error in our lifetime, often with devastating consequences. Traditionally, diagnostic decision making has been the sole responsibility of an individual clinician. However, diagnosis involves an interaction among interprofessional team members with different training, skills, cultures, knowledge, and backgrounds. Moreover, diagnostic error is prevalent in the interruption-prone environment, such as the emergency department, where the loss of information may hinder a correct diagnosis.

Objective

The overall purpose of this protocol is to improve team-based diagnostic decision making by focusing on data analytics and informatics tools that improve collective information management.

Methods

To achieve this goal, we will identify the factors contributing to failures in team-based diagnostic decision making (aim 1), understand the barriers of using current health information technology tools for team collaboration (aim 2), and develop and evaluate a collaborative decision-making prototype that can improve team-based diagnostic decision making (aim 3).

Results

Between 2019 to 2020, we are collecting data for this study. The results are anticipated to be published between 2020 and 2021.

Conclusions

The results from this study can shed light on improving diagnostic decision making by incorporating diagnostics rationale from team members. We believe a positive direction to move forward in solving diagnostic errors is by incorporating all team members, and using informatics.

International Registered Report Identifier (IRRID)

DERR1-10.2196/16047

Interactional misalignment in the UK NHS 111 healthcare telephone triage service

BACKGROUND

A recent review of primary care serious incidents suggests that diagnosis and assessment problems, underpinned by communication failures, involving the UK telephone triage service, NHS 111, may contribute to patient harm.

METHODS

The present study utilised conversation analysis to address the lack of evaluative research examining the NHS 111 system and in particular interactions between system components (call handler, computerized decision support system, patients/caller).

RESULTS

Analysis of audio recorded call interactions revealed interactional misalignment across four mapped call phases (eliciting caller details, establishing reason for call, completing the Pathways assessment, and agreeing the outcome). This misalignment has the capacity to increase the risk of system failure, particularly in relation to assessment problems and issues related to the accurate transfer of care advice. Our analysis suggests that efforts to enhance the NHS 111 system, similar telehealth services, and patient safety management more generally, should shift their focus from a limited set of individual components towards a system-specific interactionist perspective encompassing all elements.

CONCLUSIONS

Further evaluative research is required in order to build a comprehensive evidence-base concerning the multiple interacting factors influencing patient safety in the NHS 111 system.

How are distractibility and hazard prediction in driving related? Role of driving experience as moderating factor

Distraction constitute one of the 'five fatal' behaviours that contribute to road trauma, and some people may be more susceptible to it than others. It is also known that a greater ability to predict danger is related to a lower probability of suffering accidents. It could be hypothesised that drivers with a higher tendency to distraction are worse at predicting traffic hazards, but to what extent might driving experience serve to mitigate this tendency to distraction? The current study collected self-reported attentional errors from drivers by using the Attention-Related Driving Errors Scale (ARDES-Spain) in order to examine whether novice drivers suffered from inattention more than experienced drivers. The results demonstrated that novice drivers scored more highly on ARDES than experienced drivers. ARDES scores were then related to performance in a Hazard Prediction test, where participants had to report what hazard was about to happen in a series of video clips that occlude just as the hazard begins to develop. While experienced drivers were better at the Hazard Prediction test than novice drivers, those participants who reported fewer attention errors were also better able to detect the upcoming hazard following occlusion. In addition, our results demonstrate a relationship between self-reported attentional errors and the ability to predict upcoming hazards on the road, with driving experience having a moderating role. In the case of novice drivers, as their scores in the Manoeuvring Errors ARDES factor increase, their ability in Hazard Prediction diminishes, while for experienced drivers the increase is not significant. Guidance on how to improve training for drivers in order to mitigate the effects of inattention on driving safety can be addressed.

Complexity of the pediatric trauma care process: Implications for multi-level awareness

Trauma is the leading cause of disability and death in children and young adults in the US. While much is known about the medical aspects of inpatient pediatric trauma care, not much is known about the processes and roles involved in in-hospital care. Using human factors engineering (HFE) methods, we combine interview, archival document and trauma registry data to describe how intra-hospital care transitions affect process and team complexity. Specifically, we identify the 53 roles directly involved in patient care in each hospital unit and describe the 3324 total transitions between hospital units and the 69 unique pathways, from arrival to discharge, experienced by pediatric trauma patients. We continue the argument to shift from eliminating complexity to coping with it and propose supporting three levels of awareness to enhance the resilience and adaptation necessary for patient safety in health care, i.e. safety in complex systems. We discuss three levels of awareness (individual, team and organizational) and describe challenges and potential sociotechnical solutions for each. For example, one challenge to individual awareness is high time pressure. A potential solution is clinical decision support of information perception, integration and decision making. A challenge to team awareness is inadequate "non-technical" skills, e.g., leadership, communication, role clarity; simulation or another form of training could improve these. The complex, distributed nature of this process is a challenge to organizational awareness; a potential solution is to develop awareness of the process and the roles and interdependencies within it, by using process modeling or simulation.

User-defined information sharing for team situation awareness and teamwork

Team members have different roles in various scenarios to maintain situation awareness. A collaborative system should therefore provide appropriate information to the appropriate person at an appropriate time. Considering the mismatch between the designed and actually used information, this paper proposed that users should define what information to share with their team-mates. Thirty-six participants, who formed eighteen teams, used both the traditional and user-defined shared displays to perform failure diagnosis on the context of nuclear power plants. The user-defined shared display exhibited shorter diagnosis time without significant difference in correctness. Information quality, instead of quantity, was positively correlated with team mutual awareness. This study provides empirical evidence that user-defined information sharing is effective at improving operator's diagnosis performance, so the users should be able to tailor the information based on requirements.Practitioner Summary: To support team situation awareness and teamwork, the present study proposed that users should define what information to share with their team-mates. The laboratory experiment shows that user-defined information sharing shortens operator diagnosis time without degrading correctness. Information quality appears more important than information quantity in enhancing team mutual awareness.

Accounting for memes in sociotechnical systems: extending the abstraction hierarchy to consider cognitive objects

Work domain analysis (WDA) is used to model the functional structure of sociotechnical systems (STS) through the abstraction hierarchy (AH). By identifying objects, processes, functions and measures that support system purposes, WDA reveals constraints within the system. Traditionally, the AH describes system elements at the lowest level of abstraction as physical objects. Multiple analyses of complex systems reveal that many include objects that exist only at a conceptual level. This paper argues that, by extending the AH to include cognitive objects, the analytical power of WDA is extended, and novel areas of application are enabled. Three case studies are used to demonstrate the role that cognitive objects play within STS. It is concluded that cognitive objects are a valid construct that offer a significant enhancement of WDA and enable its application to some of the world's most pressing problems. Implications for future applications of WDA and the AH are discussed. Practitioner summary: Some sociotechnical systems include memes as part of their functional structure. Three case studies were used to evaluate the utility of introducing cognitive objects alongside physical ones in work domain analysis, the first phase of cognitive work analysis. Including cognitive objects increases the scope and accuracy of work domain analysis.

Distributed cognition criteria: Defined, operationalized, and applied to human-dog systems

Distributed cognition generally refers to situations in which task requirements are shared among multiple agents or, potentially, off-loaded onto the environment. With few exceptions, socially distributed cognition has largely been discussed in terms of intraspecific interactions. This conception fails to capture some forms of group-level cognition among human and non-human animals that are not readily measured or explained in mentalistic or verbal terms. In response to these limitations, we argue for a more stringent set of empirically-verifiable criteria for assessing whether a system is an instance of distributed cognition: interaction-dominant dynamics, agency, and shared task orientation. We apply this framework to humans and working dogs, and contrast the human-dog socially distributed cognitive system with humans using non-biological tools and human interaction with draft animals. The human-dog system illustrates three operationalizable factors for classifying phenomena as socially distributed cognition and extends the framework to interspecies distributed cognition.

Use of Highways in the Sky and a virtual pad for landing Head Up Display symbology to enable improved helicopter pilots situation awareness and workload in degraded visual conditions

Flight within degraded visual conditions is a great challenge to pilots of rotary-wing craft. Environmental cues typically used to guide interpretation of speed, location and approach can become obscured, forcing the pilots to rely on data available from in-cockpit instrumentation. To ease the task of flight during degraded visual conditions, pilots require easy access to flight critical information. The current study examined the effect of 'Highways in the Sky' symbology and a conformal virtual pad for landing presented using a Head Up Display (HUD) on pilots' workload and situation awareness for both clear and degraded conditions across a series of simulated rotary-wing approach and landings. Results suggest that access to the HUD lead to significant improvements to pilots' situation awareness, especially within degraded visual conditions. Importantly, access to the HUD facilitated pilot awareness in all conditions. Results are discussed in terms of future HUD development. Practitioner Summary: This paper explores the use of a novel Heads Up Display, to facilitate rotary-wing pilots' situation awareness and workload for simulated flights in both clear and degraded visual conditions. Results suggest that access to HUD facilitated pilots' situation awareness, especially when flying in degraded conditions.

Distributed cognition in aviation operations: a gate-to-gate study with implications for distributed crewing

The network analysis method, Event Analysis of Systemic Teamwork (EAST), was used to examine routine aviation operations from multiple perspectives from six key areas (i.e. Dispatch, ATC, ATM, Maintenance, Loading, and the Cockpit). Data was collected over a five-day observational field trial at an international air cargo operator. Researchers recorded the activities of agents operating within the six key areas over three outbound and two inbound flights. Three networks (i.e. social, information and task) were created for four key phases of flight: (i) pre-flight checks and engines start (ii) taxi, take-off and assent, (iii) descent, landing and taxi, and (iv) park and shut down. The networks represent a 'work audit' of short-haul cargo operations, which enabled a detailed understanding of the interactions and connections within the current system. Implications for the future of distributed crewing concepts are discussed. Practitioner Summary: An analysis of the aviation system was undertaken using the amalgamated data from three outbound and two inbound flights. These analyses show the social, information and task interactions for cargo operations. This has been used to specify requirements for future distributed crewing options.

The impact of alerting designs on air traffic controller's eye movement patterns and situation awareness

This research investigated controller' situation awareness by comparing COOPANS's acoustic alerts with newly designed semantic alerts. The results demonstrate that ATCOs' visual scan patterns had significant differences between acoustic and semantic designs. ATCOs established different eye movement patterns on fixations number, fixation duration and saccade velocity. Effective decision support systems require human-centered design with effective stimuli to direct ATCO's attention to critical events. It is necessary to provide ATCOs with specific alerting information to reflect the nature of the critical situation in order to minimise the side effects of startle and inattentional deafness. Consequently, the design of a semantic alert can significantly reduce ATCOs' response time, therefore providing valuable extra time in a time-limited situation to formulate and execute resolution strategies in critical air safety events. The findings of this research indicate that the context-specified design of semantic alerts could improve ATCO's situational awareness and significantly reduce response time in the event of Short Term Conflict Alert (STCA) activation which alerts to two aircraft having less than the required lateral or vertical separation. Practitioner Summary: Eye movements are closely linked with visual attention and can be analysed to explore shifting attention whilst performing monitoring tasks. This research has found that context-specific designed semantic alerts facilitated improved ATCO cognitive processing by integrating visual and auditory resources. Semantic designs have been demonstrated to be superior to acoustic design by directing the operator's attention more quickly to critical situations.Abbreviations: APW: area proximity warning; ASRS: aviation safety reporting system; ATC: air traffic control; ATCO: air traffic controller; ATM: air traffic management; COOPANS: cooperation between air navigation service providers; HCI: human-computer interaction; IAA: irish aviation authority; MSAW: minimum safe altitude warning; MTCD: medium-term conflict detection; SA: situation awareness; STCA: short term conflict alert; TP: trajectory prediction.

Proceed with caution: using verbal protocol analysis to measure situation awareness

Verbal protocol analysis (VPA) is often used to elicit information about the cognitive processes of operators as it provides rich data and can be used in naturalistic settings. Recently VPA has been used to investigate the acquisition and maintenance of situation awareness (SA), and to make comparisons between groups despite a lack of research regarding the efficacy of using VPA for this purpose. This train simulator experiment investigated whether VPA can effectively measure SA. Novice and expert participants were recorded on an audio device while talking aloud throughout the trials and their verbalisations were transcribed verbatim. A coding scheme developed from the transcripts was used to code the verbalisations. Results did not support the use of VPA as a measure of SA but did show that VPA detected differences in SA errors. Potential reasons for the conflicting findings between this experiment and those conducted by other researchers are discussed. Practitioner summary: This paper examined the validity of verbal protocol analysis (VPA) as a situation awareness (SA) measure. A repeated measures experiment was conducted using a train simulator. Normal VPA did not detect changes in SA but a measure of errors did. Caution should be used when using VPA to measure SA. Abbreviations: LETSSA: low-event task subjective situation awareness technique; SA: situation awareness; SAGAT: situation awareness global assessment technique; TPD: train performance display; VPA: verbal protocol analysis.

Identification of hospital cost drivers using sparse group lasso

Public hospital spending consumes a large share of government expenditure in many countries. The large cost variability observed between hospitals and also between patients in the same hospital has fueled the belief that consumption of a significant portion of this funding may result in no clinical benefit to patients, thus representing waste. Accurate identification of the main hospital cost drivers and relating them quantitatively to the observed cost variability is a necessary step towards identifying and reducing waste. This study identifies prime cost drivers in a typical, mid-sized Australian hospital and classifies them as sources of cost variability that are either warranted or not warranted-and therefore contributing to waste. An essential step is dimension reduction using Principal Component Analysis to pre-process the data by separating out the low value 'noise' from otherwise valuable information. Crucially, the study then adjusts for possible co-linearity of different cost drivers by the use of the sparse group lasso technique. This ensures reliability of the findings and represents a novel and powerful approach to analysing hospital costs. Our statistical model included 32 potential cost predictors with a sample size of over 50,000 hospital admissions. The proportion of cost variability potentially not clinically warranted was estimated at 33.7%. Given the financial footprint involved, once the findings are extrapolated nationwide, this estimation has far-reaching significance for health funding policy.

Agency modulates interactions with automation technologies

The increasing presence of automation between operators and automated systems tends to disrupt operators from action outcomes, leading them to leave the control loop. The theoretical framework of agency suggests that priming the operator about the system's upcoming behaviour could help restore an appropriate sense of control and increase user acceptance of what the system is doing. In a series of two experiments, we test whether providing information about what the system is about to do next leads to an increase in the level of user acceptance, concomitant with an increase in control and performance. Using an aircraft supervision task, we demonstrated the benefit of prime messages regarding system acceptance and performance. Taken together, our results indicate that the principles proposed by this framework could be used to improve human-machine interaction and maintain a high level of sense of control in supervisory tasks. Practitioner Summary: The out-of-the-loop performance problem is a major potential consequence of automation, leaving operators helpless to takeover automation in case of failure. Using an aircraft supervision task, the following article illustrates how the psychological approach of agency can help improving human-system interactions by designing more acceptable and more controllable automated interfaces.

Information sharing in interteam responses to disaster

Research demonstrates that information sharing is facilitated by familiarity, and having a common understanding of problems, use of lexicon, and semantic meaning. These factors can be difficult to develop within extreme environments such as disasters as members of the multi-agency system that responds often have limited experience of working together. Public inquiries repeatedly highlight the impact of information sharing difficulties on public safety, but limited academic research has focused on identifying concrete behaviours that facilitate interteam information sharing within such environments. This paper presents a case study of a national disaster response exercise involving 1,000 emergency responders. Data consist of structured observations, recordings of interteam meetings, and interviews with emergency responders. Results of mixed-method analysis indicate that interteam information sharing is delayed by limited situation awareness and poor articulation. Conversely, adopting behaviours that promote common frames for understanding interteam capabilities and information requirements improves information sharing and potentially reduces cognitive effort required to process information. Findings contribute to interteam communication theory by highlighting that in complex, time-constrained environments, having a shared understanding of responsibilities and information requirement is important for minimizing redundant deliberation and improving relevance and speed.

PRACTITIONER POINTS

Facilitating the exchange and interpretation of relevant information is important for improving situation assessment, decision-making, and the implementation of appropriate actions for addressing risks.Interteam information sharing can be particularly challenging when teams are comprised of members from across different organizations with different language and cultures that must form ad hoc to rapidly respond to problems in extreme environments.Adopting communication strategies that develop common frames-of-reference can facilitate information sharing and interteam responses to disasters.

Quantitative Analysis of Situation Awareness (QASA): modelling and measuring situation awareness using signal detection theory

This paper presents a model of situation awareness (SA) that emphasises that SA is necessarily built using a subset of available information. A technique (Quantitative Analysis of Situation Awareness - QASA), based around signal detection theory, has been developed from this model that provides separate measures of actual SA (ASA) and perceived SA (PSA), together with a feature unique to QASA, a measure of bias (information acceptance). These measures allow the exploration of the relationship between actual SA, perceived SA and information acceptance. QASA can also be used for the measurement of dynamic ASA, PSA and bias. Example studies are presented and full details of the implementation of the QASA technique are provided. Practitioner Summary: This paper presents a new model of situation awareness (SA) together with an associated tool (Quantitative Analysis of Situation Awareness - QASA) that employs signal detection theory to measure several aspects of SA, including actual and perceived SA and information acceptance. Full details are given of the implementation of the tool.

A Scientific Foundation of Simulation Games for the Analysis and Design of Complex Systems

Background. The use of simulation games for complex systems analysis and design has been acknowledged about 50 years ago. However, articles do not combine all salient factors for successful simulation games, and often stem from a clear view of one particular field of science only. With combining multiple disciplines, connect analysis and design as well as research and practice, we provide deep insights in design and use of simulation games. Aim. This article analyzes the design and evaluation process of a variety of game-based projects and activities, using existing scientific concepts and approaches, in order to establish games as a valid research tool. Our focus lies on the approach towards the use of games as design instrument; using them as an intervention in a larger, complex context, in order to design this context. With our contribution, we aim at providing insights and recommendations on the design and use of games as valid research tools, the limitations of this use, possible pitfalls, but also best practices. Method. We carried out a literature review of related work to identify the most important scientific concepts related to our approach of game design. Further use of combined quantitative and qualitative case study analyses highlights the design process and results of our own game studies. Results. The analyses yielded a consolidated conceptualization of simulation games as research instruments in complex systems analysis and design. The results also include methods for the evaluation of simulation games, additional evaluation methods, and limitations to use simulation games as research instruments. Conclusions. We propose guidelines for using simulation games as research instruments that may be of value to practitioners and scientists alike. Recommendation. We recommend practitioners and scientists to apply the guidelines presented here in their efforts to analyze and design complex systems.

Go Deeper, Go Deeper: Understanding submarine command and control during the completion of dived tracking operations

This is a world's first-of-a-kind study providing empirical evidence for understanding submarine control room performance when completing higher and lower demand Dived Tracking (DT) scenarios. A submarine control room simulator was built, using a non-commercial version of Dangerous Waters as the simulation engine. The creation of networked workstations allowed a team of nine operators to perform tasks completed by submarine command teams during DT. The Event Analysis of Systemic Teamwork (EAST) method was used to model the social, task and information networks and describe command team performance. Ten teams were recruited for the study, affording statistical comparisons of how command team roles and level of demand affected performance. Results indicate that command teams can covertly DT a contact differently depending on demand (e.g. volume of contacts). In low demand it was possible to use periscope more often than in high demand, in a 'duck-and-run' fashion. Therefore, the type of information and frequency of particular task completion, was significantly different between the higher and lower demand conditions. This resulted in different operators in the command team experiencing greater demand depending on how the DT mission objective was completed. Potential bottlenecks in the command team were identified and implications are discussed alongside suggestions for future work.