Technology as Teammate: Examining the Role of External Cognition in Support of Team Cognitive Processes

EveryBOTy Counts: Examining Human-Machine Teams in Open Source Software Development

In this study, we explore the future of work by examining differences in productivity when teams are composed of only humans or both humans and machine agents. Our objective was to characterize the similarities and differences between human and human-machine teams as they work to coordinate across their specialized roles. This form of research is increasingly important given that machine agents are becoming commonplace in sociotechnical systems and playing a more active role in collaborative work. One particular class of machine agents, bots, is being introduced to these systems to facilitate both taskwork and teamwork. We investigated the association between bots and productivity outcomes in open source software development through an analysis of hundreds of project teams. The presence of bots in teams was associated with higher levels of productivity and higher work centralization in addition to greater amounts of observed communication. The adoption of bots in software teams may have tradeoffs, in that doing so may increase productivity, but could also increase workload. We discuss the theoretical and practical implications of these findings for advancing human-machine teaming research.

Individual and team profiling to support theory of mind in artificial social intelligence

We describe an approach aimed at helping artificial intelligence develop theory of mind of their human teammates to support team interactions. We show how this can be supported through the provision of quantifiable, machine-readable, a priori information about the human team members to an agent. We first show how our profiling approach can capture individual team member characteristic profiles that can be constructed from sparse data and provided to agents to support the development of artificial theory of mind. We then show how it captures features of team composition that may influence team performance. We document this through an experiment examining factors influencing the performance of ad-hoc teams executing a complex team coordination task when paired with an artificial social intelligence (ASI) teammate. We report the relationship between the individual and team characteristics and measures related to task performance and self-reported perceptions of the ASI. The results show that individual and emergent team profiles were able to characterize features of the team that predicted behavior and explain differences in perceptions of ASI. Further, the features of these profiles may interact differently when teams work with human versus ASI advisors. Most strikingly, our analyses showed that ASI advisors had a strong positive impact on low potential teams such that they improved the performance of those teams across mission outcome measures. We discuss these findings in the context of developing intelligent technologies capable of social cognition and engage in collaborative behaviors that improve team effectiveness.

From Teams to Teamness: Future Directions in the Science of Team Cognition

OBJECTIVE

We review the current state-of-the-art in team cognition research, but more importantly describe the limitations of existing theories, laboratory paradigms, and measures considering the increasing complexities of modern teams and the study of team cognition.

BACKGROUND

Research on, and applications of, team cognition has led to theories, data, and measures over the last several decades.

METHOD

This article is based on research questions generated in a spring 2022 seminar on team cognition at Arizona State University led by the first author.

RESULTS

Future research directions are proposed for extending the conceptualization of teams and team cognition by examining dimensions of teamness; extending laboratory paradigms to attain more realistic teaming, including nonhuman teammates; and advancing measures of team cognition in a direction such that data can be collected unobtrusively, in real time, and automatically.

CONCLUSION

The future of team cognition is one of the new discoveries, new research paradigms, and new measures.

APPLICATION

Extending the concepts of teams and team cognition can also extend the potential applications of these concepts.

Team Dynamics and Collaborative Problem-Solving for Lunar Construction: Lessons From Complex Construction Scenarios on Earth

OBJECTIVE

This paper surveys the existing literature surrounding problem-solving and team dynamics in complex and unpredictable scenarios, and evaluates the applicability of studying Earth-based construction teams to identify training needs for Lunar construction crews.

BACKGROUND

Lunar and other space exploration construction crews will work in extreme environments and face unpredictable challenges, necessitating real-time problem-solving to address unexpected contingencies. This work will require coordination with Mission Control and autonomous assistants, so crew training must account for multi-agent, distributed teamwork.

METHOD

A narrative literature review identified processes, attributes, and skills necessary for the success of Lunar construction teams. We summarized relevant frameworks and synthesized collective findings into over-arching trends and remaining research gaps.

RESULTS

While significant literature exists surrounding team performance, very little systematic inquiry has been done with a focus on Lunar construction crews and operations, particularly with respect to dynamic problem-solving and team-based decision-making. Established and standardized metrics for evaluating team performance are lacking, resulting in significant variation in reported outcomes between studies.

CONCLUSION

Lunar and other space exploration construction teams will need training that focuses on developing the right approach to team-based problem-solving, rather than on preparing response execution for known contingencies. An investigation of successful Earth-based construction crews may facilitate the development of relevant metrics for training future Lunar construction crews.

APPLICATION

Metrics and team training protocols developed for future Lunar construction teams may be adaptable and applicable to a wide range of extreme teams facing uncertain challenges, such as aircrews, surgical teams, first responders, and construction crews.

Fluid teams in the metaverse: exploring the (un)familiar

The metaverse is a new and evolving environment for fluid teams and their coordination in organizations. Fluid teams may have no prior familiarity with each other or working together. Yet fluid teams are known to benefit from a degree of familiarity-knowledge about teams, members, and working together-in team coordination and performance. The metaverse is unfamiliar territory that promises fluidity in contexts-seamless traversal between physical and virtual worlds. This fluidity in contexts has implications for familiarity in interaction, identity, and potentially time. We explore the opportunities and challenges that the metaverse presents in terms of (un)familiarity. Improved understandings of (un)familiarity may pave the way for new forms of fluid team experiences and uses.

The Impact of Information Integration in a Simulation of Future Submarine Command and Control

OBJECTIVE

Examine the extent to which increasing information integration across displays in a simulated submarine command and control room can reduce operator workload, improve operator situation awareness, and improve team performance.

BACKGROUND

In control rooms, the volume and number of sources of information are increasing, with the potential to overwhelm operator cognitive capacity. It is proposed that by distributing information to maximize relevance to each operator role (increasing information integration), it is possible to not only reduce operator workload but also improve situation awareness and team performance.

METHOD

Sixteen teams of six novice participants were trained to work together to combine data from multiple sensor displays to build a tactical picture of surrounding contacts at sea. The extent that data from one display were available to operators at other displays was manipulated (information integration) between teams. Team performance was assessed as the accuracy of the generated tactical picture.

RESULTS

Teams built a more accurate tactical picture, and individual team members had better situation awareness and lower workload, when provided with high compared with low information integration.

CONCLUSION

A human-centered design approach to integrating information in command and control settings can result in lower workload, and enhanced situation awareness and team performance.

APPLICATION

The design of modern command and control rooms, in which operators must fuse increasing volumes of complex data from displays, may benefit from higher information integration based on a human-centered design philosophy, and a fundamental understanding of the cognitive work that is carried out by operators.

Assessing Team Effectiveness by How Players Structure Their Search in a First-Person Multiplayer Video Game

People working as a team can achieve more than when working alone due to a team's ability to parallelize the completion of tasks. In collaborative search tasks, this necessitates the formation of effective division of labor strategies to minimize redundancies in search. For such strategies to be developed, team members need to perceive the task's relevant components and how they evolve over time, as well as an understanding of what others will do so that they can structure their own behavior to contribute to the team's goal. This study explored whether the capacity for team members to coordinate effectively can be related to how participants structure their search behaviors in an online multiplayer collaborative search task. Our results demonstrated that the structure of search behavior, quantified using detrended fluctuation analysis, was sensitive to contextual factors that limit a participant's ability to gather information. Further, increases in the persistence of movement fluctuations during search behavior were found as teams developed more effective coordinative strategies and were associated with better task performance.

Human-Autonomy Teaming: A Review and Analysis of the Empirical Literature

OBJECTIVE

We define human-autonomy teaming and offer a synthesis of the existing empirical research on the topic. Specifically, we identify the research environments, dependent variables, themes representing the key findings, and critical future research directions.

BACKGROUND

Whereas a burgeoning literature on high-performance teamwork identifies the factors critical to success, much less is known about how human-autonomy teams (HATs) achieve success. Human-autonomy teamwork involves humans working interdependently toward a common goal along with autonomous agents. Autonomous agents involve a degree of self-government and self-directed behavior (agency), and autonomous agents take on a unique role or set of tasks and work interdependently with human team members to achieve a shared objective.

METHOD

We searched the literature on human-autonomy teaming. To meet our criteria for inclusion, the paper needed to involve empirical research and meet our definition of human-autonomy teaming. We found 76 articles that met our criteria for inclusion.

RESULTS

We report on research environments and we find that the key independent variables involve autonomous agent characteristics, team composition, task characteristics, human individual differences, training, and communication. We identify themes for each of these and discuss the future research needs.

CONCLUSION

There are areas where research findings are clear and consistent, but there are many opportunities for future research. Particularly important will be research that identifies mechanisms linking team input to team output variables.

Automation as an equal team player for humans? - A view into the field and implications for research and practice

The practical reality and feasibility of Human-Autonomy Teaming (HAT) are analyzed from an experts' point of view, considering current possibilities of various fields. We aim to find out whether the topics discussed scientifically are also practically relevant, to identify requirements for successful HAT, and to derive further research needs. Intensive guideline-based interviews with 28 experts from different industries are conducted and compared to the results of our literature review. The topics discussed scientifically are also practically relevant. Today's technology is far from being able to meet the practical requirements for successful HAT, as postulated in the literature. Contrary to the Human-Automation Interaction, the concept of HAT is hardly applied in the field. Identified key aspects for successful HAT are converted into a model. Future research needs with practical impact exist especially in the area of heterarchy, system knowledge, anticipation of mental states, and consideration of human needs and emotions.

Are you there? Presence in collaborative distance work

Already before the pandemic, digitally mediated collaborative work and communication were perceived as challenging. We investigate the attitudes towards emerging technologies and for transforming practises in workplaces. The focus lies on understanding the readiness for appropriating emotional tracking on presence and support for collaboration. The research-based design framework allowed to combine the various perspectives of the transdisciplinary team. Methods included participatory design, design thinking, contextual inquiry and prototype testing for enhancing presence while working with shared objects in video conferencing to explore the appropriation of tools. The findings revealed four indications: 1) awareness of interlocutors’ presence during synchronous communication is crucial. 2)  Emotion and behaviour tracking raises concerns about privacy and personal control over what is displayed to others, and technology could be simpler non-distracting the work at hand.  3) The prototype was found to enhance the feeling of presence without disturbing work at hand, and 4) appropriation requires a step-by-step approach.

Cognition in crisis management teams: an integrative analysis of definitions

In large-scale extreme events, multidisciplinary crisis management teams (CMTs) are required to function together cognitively. Despite theoretical maturity in team cognition and recurrent emphasis on cognition in the crisis management practices, no synthesis of theoretical and practical discourses is currently available, limiting empirical investigations of cognition in CMTs. To address this gap, this paper aims to review the definitions of cognition in CMTs, with a particular focus on examining if and to what extent they are diversified. Through a systematic process to search peer-reviewed journal articles published in English from 1990 to 2019, 59 articles were selected with 62 coded definitions of 11 different constructs. The similarities and variabilities of the definitions were examined in terms of their theoretical and practical emphases and then synthesised into an integrative definition expected to serve as a general guide of reference for future researchers seeking an operational definition of cognition in CMTs. Practitioner summary: Understanding of cognition in CMTs is grounded in various theories and models with varying assumptions. An integrative conceptualisation of such cognition as interaction within and across CMTs to perceive, diagnose, and adapt to the crisis may facilitate the accumulation of knowledge and future operationalisations. Abbreviations: CMT(s): crisis management team(s); SMM: shared mental model; TMM: team mental model; COP: common operating picture; SSA: shared situation awareness; TSA: team situation awareness; DC: distributed cognition; ITC: interactive team cognition; TMS: transitive memory system(s); DSA: distributed situation awareness.

GEM-NET: Lessons in Multi-Institution Teamwork Using Collaboration Software

The Center for Genetically Encoded Materials (C-GEM) is an NSF Phase I Center for Chemical Innovation that comprises six laboratories spread across three university campuses. Our success as a multi-institution research team demanded the development of a software infrastructure, GEM-NET, that allows all C-GEM members to work together seamlessly-as though everyone was in the same room. GEM-NET was designed to support both science and communication by integrating task management, scheduling, data sharing, and collaborative document and code editing with frictionless internal and public communication; it also maintains security over data and internal communications. In this Article, we document the design and implementation of GEM-NET: our objectives and motivating goals, how each component contributes to these goals, and the lessons learned throughout development. We also share open source code for several custom applications and document how GEM-NET can benefit users in multiple fields and teams that are both small and large. We anticipate that this knowledge will guide other multi-institution teams, regardless of discipline, to plan their software infrastructure and utilize it as swiftly and smoothly as possible.

Fighter pilots' teamwork: a descriptive study

The execution of teamwork varies widely depending on the domain and task in question. Despite the considerable diversity of teams and their operation, researchers tend to aim for unified theories and models regardless of field. However, we argue that there is a need for translation and adaptation of the theoretical models to each specific domain. To this end, a case study was carried out on fighter pilots and it was investigated how teamwork is performed in this specialised and challenging environment, with a specific focus on the dependence on technology for these teams. The collaboration between the fighter pilots is described and analysed using a generic theoretical model for effective teamwork from the literature. The results show that domain-specific application and modification is needed in order for the model to capture fighter pilot's teamwork. The study provides deeper understanding of the working conditions for teams of pilots and gives design implications for how tactical support systems can enhance teamwork in the domain. Practitioner summary: This article presents a qualitative interview study with fighter pilots based on a generic theoretical teamwork model applied to the fighter domain. The purpose is to understand the conditions under which teams of fighter pilots work and to provide guidance for the design of future technological aids.

Advancing the Science of Collaborative Problem Solving

Collaborative problem solving (CPS) has been receiving increasing international attention because much of the complex work in the modern world is performed by teams. However, systematic education and training on CPS is lacking for those entering and participating in the workforce. In 2015, the Programme for International Student Assessment (PISA), a global test of educational progress, documented the low levels of proficiency in CPS. This result not only underscores a significant societal need but also presents an important opportunity for psychological scientists to develop, adopt, and implement theory and empirical research on CPS and to work with educators and policy experts to improve training in CPS. This article offers some directions for psychological science to participate in the growing attention to CPS throughout the world. First, it identifies the existing theoretical frameworks and empirical research that focus on CPS. Second, it provides examples of how recent technologies can automate analyses of CPS processes and assessments so that substantially larger data sets can be analyzed and so students can receive immediate feedback on their CPS performance. Third, it identifies some challenges, debates, and uncertainties in creating an infrastructure for research, education, and training in CPS. CPS education and assessment are expected to improve when supported by larger data sets and theoretical frameworks that are informed by psychological science. This will require interdisciplinary efforts that include expertise in psychological science, education, assessment, intelligent digital technologies, and policy.

Exploring Mentoring in the Context of Team Science

Despite an increasing amount of research into team science and collaboration, little is known about post-doctoral scholars' pathways to becoming independent researchers, particularly as it relates to the mentoring relationship. The purpose of our study was to explore if and how the team science framework promotes collaboration and the development of independent researchers from mentees' and mentors' perspectives. A convenience sample of faculty mentors and postdoctoral mentees, participated in semi-structured interviews. The analysis revealed three themes for the mentees: working with others, evaluating mentoring, exploring resources, and five themes for the mentors: elaborating on scholarly productivity, mentoring style, setting team culture, strategies to develop T32 scholars, and evaluating program qualities. The findings showed that scholars were engaged in an atmosphere of collaboration fostered by their mentors and that mentors encouraged scholars' capacity for scientific thinking. Future studies should acquire the perspectives of all team members to examine how teams function.

Complex Problem Solving in Assessments of Collaborative Problem Solving

Collaborative problem solving (ColPS) proficiency was developed as a new assessment for the Programme for International Student Assessment (PISA) in the 2015 international evaluation of student skills and knowledge. The assessment framework defined by the PISA ColPS 2015 expert group crossed three major collaboration processes with four problem solving processes that were adopted from the PISA 2012 individual problem solving assessment to form a matrix of 12 specific skills. The three major collaboration processes are (1) establishing and maintaining shared understanding; (2) taking appropriate action; and (3) establishing and maintaining team organization. The four problem solving processes are exploring and understanding the problem, representing and formulating the problem, planning and executing strategies, and monitoring and reflecting on the problem-solving activities. This article discusses how the problem-solving dimension was integrated with the collaboration dimension. We also discuss how computer agents were involved in the PISA ColPS 2015 assessment in order to ensure a satisfactory assessment of collaborative problem solving. Examples of the use of agents to assess ColPS are provided in the context of a released PISA item and a project conducted in Taiwan.

Problem-Solving Phase Transitions During Team Collaboration

Multiple theories of problem-solving hypothesize that there are distinct qualitative phases exhibited during effective problem-solving. However, limited research has attempted to identify when transitions between phases occur. We integrate theory on collaborative problem-solving (CPS) with dynamical systems theory suggesting that when a system is undergoing a phase transition it should exhibit a peak in entropy and that entropy levels should also relate to team performance. Communications from 40 teams that collaborated on a complex problem were coded for occurrence of problem-solving processes. We applied a sliding window entropy technique to each team's communications and specified criteria for (a) identifying data points that qualify as peaks and (b) determining which peaks were robust. We used multilevel modeling, and provide a qualitative example, to evaluate whether phases exhibit distinct distributions of communication processes. We also tested whether there was a relationship between entropy values at transition points and CPS performance. We found that a proportion of entropy peaks was robust and that the relative occurrence of communication codes varied significantly across phases. Peaks in entropy thus corresponded to qualitative shifts in teams' CPS communications, providing empirical evidence that teams exhibit phase transitions during CPS. Also, lower average levels of entropy at the phase transition points predicted better CPS performance. We specify future directions to improve understanding of phase transitions during CPS, and collaborative cognition, more broadly.