The role of trainee knowledge structures in aviation team environments

Measurement and performance impact of team mental models on process performance

Efficient business process execution is an essential part of an organisation’s success. It depends on good dynamic decision making of process actors that is guided by their mental models of business processes (MMBP). The study investigates the effect of MMBPs on process performance at two levels. At the level of individuals, the impact of MMBP accuracy on performance is analyzed, and at the level of a team, the effect of similarity of MMBPs of all team on performance is researched. At both levels, MMBPs are differentiated in a narrow part that focuses on the mental representations of process steps that precede or follow on the one conducted by the actor and a holistic model that captures the process as a whole. We use laboratory observations with 159 participants in 10 teams from a real effort loan processing role play. We obtain individual MMBP accuracy measures by using a process knowledge test and measure the process performance of teams with the outcome of the role play. Our study contributes in three ways to existing research. First, the measurement approach of individual MMBP accuracy and similarity is extended to the level of teams. Second, the study shows that the accuracy of both narrow and holistic MMBPs as well as similarity of holistic MMBPs positively impact team process performance. Third, by using an observable team process performance measures from a real-effort task, we increase the validity of our findings compared to other research relying on self-assessed performance measures.

Looking through the eyes of the multidisciplinary team: the design and clinical evaluation of a decision support system for lung cancer care

Background

Decision-making in lung cancer is complex due to a rapidly increasing amount of diagnostic data and treatment options. The need for timely and accurate diagnosis and delivery of care demands high-quality multidisciplinary team (MDT) collaboration and coordination. Clinical decision support systems (CDSSs) can potentially support MDTs in constructing a shared mental model of a patient case. This enables the team to assess the strength and completeness of collected diagnostic data, stratification for the right personalized therapy driven by clinical stage and other treatment-influencing factors, and adapt care management strategies when needed. Current CDSSs often have a suboptimal fit into the decision-making workflow, which hampers their impact in clinical practice.

Methods

A CDSS for multidisciplinary decision-making in lung cancer was designed to support the abovementioned goals through presentation of relevant clinical data in line with existing mental model structures of the MDT members. The CDSS was tested in a simulated multidisciplinary tumor board meeting for primary diagnosis and treatment selection, based on de-identified primary lung cancer cases (n=8). Decision course analysis, eye-tracking data and questionnaires were used to assess the impact of the CDSS on constructing shared mental models to improve the decision-making process and outcome.

Results

The CDSS supported the team in their self-correcting capacity for accurate diagnosis and TNM classification. It enabled cross-validation of diagnostic findings, surfaced discordance between diagnostic tests and facilitated cancer staging according the diagnostic evidence, as well as spotting contra-indications for personalized treatment selection.

Conclusions

This study shows the potential of CDSS on clinical decision making, when these systems are properly designed in line with clinical thinking. The presented setup enables assessment of the impact of CDSS design on clinical decision making and optimization of CDSSs to maximize their effect on decision quality and confidence.

Optical Illusions and Spatial Disorientation in Aviation Pilots

Optical illusions are involved in the perception of false or erroneous images which might involve disorientation. They occur by a discordance by the peripheral systems about the information captured and generally, resulting in pilots failure to recognize key signals. The aim of this study is to review the state of the art of spatial disorientation and optical illusions in aviation pilots. This kind of disorientation has important practical consequences, because a remarkable percentage of plane accidents are related to pilot's optical illusions. An exhaustive review using pubmed and semantic scholar databases was conducted to find out the most frequent optical illusions in aviation pilots. A total of 45 full text articles published English or Spanish were reviewed. To our knowledge, this is the first study to review exhaustively and describe the main factors involved in spatial disorientation and optical illusions affecting aviation pilots. Mainly, contextual factors: width of landing track lights, nocturnal operations or low visibility, inclination of the landing track, decline of the ground, size of habitual references, low level approach on the water, black hole, sky/terrain confusion, distortion by climatic factors, autokinesis or autocinetics, optional investment illusion, illusions by vection, false horizon, rain on the windshield, misalignment in the approach, vibrations, somatogravic illusion, coriolis illusion and "G" forces. In a lesser extent, human factors and pathologies of the visual systems involved in spatial disorientation and associated optical illusions affecting aviation pilots are also described.

DISCUSSION

Practical implications are further discussed.

Crew Resource Management Training Research, Practice, and Lessons Learned

Crew resource management (CRM) training was introduced to the aviation community in 1979. Since then it has evolved and matured and is now being applied in a number of domains, including health care and offshore oil production. There is abundant literature resulting from research in the area, but there is no recent comprehensive review of the origins, current state, and future direction of CRM. The purpose of this chapter is to perform that review and provide the reader with an understanding of the research, practice, and training associated with CRM. We also provide a number of lessons learned based on the literature and our observations, as well as future needs of the community.

Safety management practices and safety behaviour: assessing the mediating role of safety knowledge and motivation

Safety management practices not only improve working conditions but also positively influence employees' attitudes and behaviours with regard to safety, thereby reducing accidents in workplace. This study measured employees' perceptions on six safety management practices and self-reported safety knowledge, safety motivation, safety compliance and safety participation by conducting a survey using questionnaire among 1566 employees belonging to eight major accident hazard process industrial units in Kerala, a state in southern part of India. The reliability and unidimesionality of all the scales were found acceptable. Path analysis using AMOS-4 software showed that some of the safety management practices have direct and indirect relations with the safety performance components, namely, safety compliance and safety participation. Safety knowledge and safety motivation were found to be the key mediators in explaining these relationships. Safety training was identified as the most important safety management practice that predicts safety knowledge, safety motivation, safety compliance and safety participation. These findings provide valuable guidance for researchers and practitioners for identifying the mechanisms by which they can improve safety of workplace.

The development, design, testing, refinement, simulation and application of an evaluation framework for communities of practice and social-professional networks

BACKGROUND

Communities of practice and social-professional networks are generally considered to enhance workplace experience and enable organizational success. However, despite the remarkable growth in interest in the role of collaborating structures in a range of industries, there is a paucity of empirical research to support this view. Nor is there a convincing model for their systematic evaluation, despite the significant potential benefits in answering the core question: how well do groups of professionals work together and how could they be organised to work together more effectively? This research project will produce a rigorous evaluation methodology and deliver supporting tools for the benefit of researchers, policymakers, practitioners and consumers within the health system and other sectors. Given the prevalence and importance of communities of practice and social networks, and the extent of investments in them, this project represents a scientific innovation of national and international significance.

METHODS AND DESIGN

Working in four conceptual phases the project will employ a combination of qualitative and quantitative methods to develop, design, field-test, refine and finalise an evaluation framework. Once available the framework will be used to evaluate simulated, and then later existing, health care communities of practice and social-professional networks to assess their effectiveness in achieving desired outcomes. Peak stakeholder groups have agreed to involve a wide range of members and participant organisations, and will facilitate access to various policy, managerial and clinical networks.

DISCUSSION

Given its scope and size, the project represents a valuable opportunity to achieve breakthroughs at two levels; firstly, by introducing novel and innovative aims and methods into the social research process and, secondly, through the resulting evaluation framework and tools. We anticipate valuable outcomes in the improved understanding of organisational performance and delivery of care. The project's wider appeal lies in transferring this understanding to other health jurisdictions and to other industries and sectors, both nationally and internationally. This means not merely publishing the results, but contextually interpreting them, and translating them to advance the knowledge base and enable widespread institutional and organisational application.

Mental models: an alternative evaluation of a sensemaking approach to ethics instruction

In spite of the wide variety of approaches to ethics training it is still debatable which approach has the highest potential to enhance professionals' integrity. The current effort assesses a novel curriculum that focuses on metacognitive reasoning strategies researchers use when making sense of day-to-day professional practices that have ethical implications. The evaluated trainings effectiveness was assessed by examining five key sensemaking processes, such as framing, emotion regulation, forecasting, self-reflection, and information integration that experts and novices apply in ethical decision-making. Mental models of trained and untrained graduate students, as well as faculty, working in the field of physical sciences were compared using a think-aloud protocol 6 months following the ethics training. Evaluation and comparison of the mental models of participants provided further validation evidence for sensemaking training. Specifically, it was found that trained students applied metacognitive reasoning strategies learned during training in their ethical decision-making that resulted in complex mental models focused on the objective assessment of the situation. Mental models of faculty and untrained students were externally-driven with a heavy focus on autobiographical processes. The study shows that sensemaking training has a potential to induce shifts in researchers' mental models by making them more cognitively complex via the use of metacognitive reasoning strategies. Furthermore, field experts may benefit from sensemaking training to improve their ethical decision-making framework in highly complex, novel, and ambiguous situations.

An action research protocol to strengthen system-wide inter-professional learning and practice [LP0775514]

BACKGROUND

Inter-professional learning (IPL) and inter-professional practice (IPP) are thought to be critical determinants of effective care, improved quality and safety and enhanced provider morale, yet few empirical studies have demonstrated this. Whole-of-system research is even less prevalent. We aim to provide a four year, multi-method, multi-collaborator action research program of IPL and IPP in defined, bounded health and education systems located in the Australian Capital Territory (ACT). The project is funded by the Australian Research Council under its industry Linkage Program.

METHODS/DESIGN

The program of research will examine in four inter-related, prospective studies, progress with IPL and IPP across tertiary education providers, professional education, regulatory and registration bodies, the ACT health system's streams of care activities and teams, units and wards of the provider facilities of the ACT health system. One key focus will be on push-pull mechanisms, ie, how the education sector creates student-enabled IPP and the health sector demands IPL-oriented practitioners. The studies will examine four research aims and meet 20 research project objectives in a comprehensive evaluation of ongoing progress with IPL and IPP.

DISCUSSION

IPP and IPL are said to be cornerstones of health system reforms. We will measure progress across an entire health system and the clinical and professional education systems that feed into it. The value of multi-methods, partnership research and a bi-directional push-pull model of IPL and IPP will be tested. Widespread dissemination of results to practitioners, policymakers, managers and researchers will be a key project goal.

Relationships among team ability composition, team mental models, and team performance

This study examined the relationship between the similarity and accuracy of team mental models and compared the extent to which each predicted team performance. The relationship between team ability composition and team mental models was also investigated. Eighty-three dyadic teams worked on a complex skill task in a 2-week training protocol. Results indicated that although similarity and accuracy of team mental models were significantly related, accuracy was a stronger predictor of team performance. In addition, team ability was more strongly related to the accuracy than to the similarity of team mental models and accuracy partially mediated the relationship between team ability and team performance, but similarity did not.

Investigating Linear and Interactive Effects of Shared Mental Models on Safety and Efficiency in a Field Setting

Linkages between 2 types of shared mental models (SMMs)--that is, positional-goal interdependencies and cue-strategy associations--and effectiveness in an air traffic control environment were investigated. Two types of SMMs were expected to contribute uniquely, as well as interact, to predict tower safety and efficiency. Using SMM data from 306 air traffic controllers, and corresponding archival efficiency and safety measures for 47 airports, the authors found no significant linear relationships between SMMs and either outcome measure. However, the 2 SMMs interacted with one another to predict both outcomes. Results are discussed in terms of the importance of measuring multiple types of SMMs, the examination of complex relationships, and the importance of indexing decisions.

A Meta-Analytic Review of Behavior Modeling Training

A meta-analysis of 117 studies evaluated the effects of behavior modeling training (BMT) on 6 training outcomes, across characteristics of training design. BMT effects were largest for learning outcomes, smaller for job behavior, and smaller still for results outcomes. Although BMT effects on declarative knowledge decayed over time, training effects on skills and job behavior remained stable or even increased. Skill development was greatest when learning points were used and presented as rule codes and when training time was longest. Transfer was greatest when mixed (negative and positive) models were presented, when practice included trainee-generated scenarios, when trainees were instructed to set goals, when trainees' superiors were also trained, and when rewards and sanctions were instituted in trainees' work environments.

Effects of Training Goals and Goal Orientation Traits on Multidimensional Training Outcomes and Performance Adaptability

This research examined the effects of mastery vs. performance training goals and learning and performance goal orientation traits on multidimensional outcomes of training. Training outcomes included declarative knowledge, knowledge structure coherence, training performance, and self-efficacy. We also examined the unique impact of the training outcomes on performance adaptability by predicting generalization to a more difficult and complex version of the task. The experiment involved 60 trainees learning a complex computer simulation over 2 days. The research model posited independent effects for training goals relative to goal orientation traits and independent contributions of training outcomes to the performance adaptability of trainees. The findings were consistent with the proposed model. In particular, self-efficacy and knowledge structure coherence made unique contributions to the prediction of performance adaptability after controlling for prior training performance and declarative knowledge. Implications and extensions are discussed. Copyright 2001 Academic Press.

Team training in the skies: does crew resource management (CRM) training work?

The aviation community has invested great amounts of money and effort into crew resource management (CRM) training. Using D. L. Kirkpatrick's (1976) framework for evaluating training, we reviewed 58 published accounts of CRM training to determine its effectiveness within aviation. Results indicated that CRM training generally produced positive reactions, enhanced learning, and promoted desired behavioral changes. However, we cannot ascertain whether CRM has an effect on an organization's bottom line (i.e., safety). We discuss the state of the literature with regard to evaluation of CRM training programs and, as a result, call for the need to conduct systematic, multilevel evaluation efforts that will show the true effectiveness of CRM training. As many evaluations do not collect data across levels (as suggested by D. L. Kirkpatrick, 1976, and by G. M. Alliger, S. I. Tannenbaum, W. Bennett, Jr., & H. Traver, 1997), the impact of CRM cannot be truly determined; thus more and better evaluations are needed and should be demanded.

The influence of shared mental models on team process and performance

The influence of teammates' shared mental models on team processes and performance was tested using 56 undergraduate dyads who "flew" a series of missions on a personal-computer-based flight-combat simulation. The authors both conceptually and empirically distinguished between teammates' task- and team-based mental models and indexed their convergence or "sharedness" using individually completed paired-comparisons matrices analyzed using a network-based algorithm. The results illustrated that both shared-team- and task-based mental models related positively to subsequent team process and performance. Furthermore, team processes fully mediated the relationship between mental model convergence and team effectiveness. Results are discussed in terms of the role of shared cognitions in team effectiveness and the applicability of different interventions designed to achieve such convergence.

The design and delivery of crew resource management training: exploiting available resources

Despite widespread acceptance throughout commercial and military settings, crew resource management (CRM) training programs have not escaped doubts about their effectiveness. The current state of CRM training is an example of how an entire body of pertinent research and development has not had the impact on practice that it could. In this paper we outline additional resources (i.e., principles, information, findings, and guidelines) from the team training and training effectiveness research literatures that can be used to improve the design and delivery of CRM training. Some of the resources discussed include knowledge about training effectiveness, training teamwork-related skills, scenario design, and performance measurement. We conclude with a discussion of emerging resources as well as those that need to be developed. The purpose of this paper is to provide the CRM training developer with better access to resources that can be applied to the design and delivery of CRM training programs.