The role of expertise research and human factors in capturing, explaining, and producing superior performance

Interpretation, confidence and application of the standardised terms: Identified, Probable, Possible, Exclude and Insufficient in forensic odontology identification

Forensic odontology identification scales are used to express certainty of identifications of deceased persons. These standardized scales are assumed to convey unambiguous expert opinions and facilitate communication between forensic odontologists and end users. However, to date no studies have investigated how the experts interpret and use these scales. Forensic odontology identification scales are used to express certainty of identifications of deceased persons. These standardized scales are assumed to convey unambiguous expert opinions and facilitate communication between forensic odontologists and end users. However, to date no studies have investigated how the experts interpret and use these scales. This paper aims to examine the interpretation of the DVISYS forensic identification scale and choices of the levels in the scale subsequent to, and derived from, comparison of pairs of dental radiographs by extending the analysis of the data collected in the study by Page and Lain et. al. 2017. The studied variables: self-reported confidence, forced binary decision of match and non-match, choice of level in the DVISYS scale (Identified, Probable, Possible, Insufficient and Exclude) were further analysed in this study using mixed models for relationships between the choices of level in the identification scale and the fundamental beliefs of likelihood of identification. The results of this further analysis showed that the reported confidence of the decisions was correlated to the difficulty of cases, and as confidence decreased the use of less definitive terms ('Probable', 'Possible' and 'Insufficient') increased. 'Probable' and 'Possible' were used mainly in underlying beliefs below that of 'Identified' whereas 'Insufficient' was used mainly to convey a sublevel of 'Exclude'. The use of 'Insufficient' in this study was not consistent with the prescribed definition of the term. The participants of the original study were not aware of the difficulty grading of the cases nor were required to grade them, however the reported confidence was systematically correlated to difficulty. Furthermore, indicated confidence level was correlated with choice of level on the scale in general, but the interpretation of the definition and application of the terms varied. The findings reported here contribute to the foundational knowledge of factors governing the interpretation and application of the DVISYS forensic odontology identification scale and suggest that this scale may need to be modified.

In Search of Characterizing Surgical Skill

OBJECTIVE

This paper provides a literature review and detailed discussion of surgical skill terminology. Culminating in a novel model that proposes a set of unique definitions, this review is designed to facilitate shared understanding to study and develop metrics quantifying surgical skill.

DESIGN

Objective surgical skill analysis depends on consistent definitions and shared understanding of terms like performance, expertise, experience, aptitude, ability, competency, and proficiency.

STRUCTURE

Each term is discussed in turn, drawing from existing literature and colloquial uses.

IMPLICATIONS

A new model of definitions is proposed to cement a common and consistent lexicon for future skills analysis, and to quantitatively describe a surgeon's performance throughout their career.

Esports: The Chess of the 21st Century

For many decades, researchers have explored the true potential of human achievement. The expertise field has come a long way since the early works of de Groot (1965) and Chase and Simon (1973). Since then, this inquiry has expanded into the areas of music, science, technology, sport, academia, and art. Despite the vast amount of research to date, the capability of study methodologies to truly capture the nature of expertise remains questionable. Some considerations include (i) the individual bias in the retrospective recall of developmental activities, (ii) the ability to develop ecologically valid tasks, and (iii) difficulties capturing the influence of confounding factors on expertise. This article proposes that expertise research in electronic sports (esports) presents an opportunity to overcome some of these considerations. Esports involves individuals or teams of players that compete in video game competitions via human-computer interaction. Advantages of applying the expert performance approach in esports include (i) developmental activities are objectively tracked and automatically logged online, (ii) the constraints of representative tasks correspond with the real-world environment of esports performance, and (iii) expertise has emerged without the influence of guided systematic training environments. Therefore, this article argues that esports research provides an ideal opportunity to further advance research on the development and assessment of human expertise.

Enhancing the Impact of Mobile Health Literacy Interventions to Reduce Health Disparities

Health literacy is a key factor in health outcomes that should be considered when creating mobile health promotion apps. In this paper, we detail our work over the past 10 years in developing the theory and practice of targeting the content of mobile apps at a level appropriate for the intended audience. We include a review of our theory of health literacy as expertise, the ASK model, and integrate it with the Theory of Planned Behavior. We then provide data that support both the model and its use. More recently, we have developed a predictive analytic model that uses demographic information and patient performance on a 10-item screening measure to determine patient level of health literacy at a high level of accuracy. The predictive model will enable apps to automatically provide content to users at an appropriate level of health literacy. This strategy, along with other aspects of tailoring, will allow apps to be more personally relevant to users, enhancing their effects in promoting health behavior change.

Investigating Neural Sensorimotor Mechanisms Underlying Flight Expertise in Pilots: Preliminary Data From an EEG Study

Over the last decade, the efforts toward unraveling the complex interplay between the brain, body, and environment have set a promising line of research that utilizes neuroscience to study human performance in natural work contexts such as aviation. Thus, a relatively new discipline called neuroergonomics is holding the promise of studying the neural mechanisms underlying human performance in pursuit of both theoretical and practical insights. In this work, we utilized a neuroergonomic approach by combining insights from ecological psychology and embodied cognition to study flight expertise. Specifically, we focused on the Mirror Neuron system as a key correlate for understanding the interaction between an individual and the environment, suggesting that it can be used to index changes in the coupling of perception-action associated with skill development. In this study, we measured the EEG mu suppression as a proxy of the Mirror Neuron system in experts (pilots) and novices while performing a distance estimation task in a landing scenario. To survey the specificity of this measure, we considered central, parietal and occipital electrode pools and analyzed alpha (8–13 Hz) and beta (18–25 Hz) rhythm bands. We hypothesized that in experts vs. novices, specific neural sensorimotor brain activity would underpin the connection between perception and action in an in-flight context. Preliminary results indicate that alpha and beta rhythm suppression was area-specific irrespective of groups, present in the central electrodes placed over the motor areas. Group analysis revealed that specifically alpha mu rhythm, but not beta, was significantly more suppressed in pilots vs. novices. Complementing these findings we found a trend in which the strength of mu suppression increased with the sense of presence experienced by the pilots. Such sensorimotor activation is in line with the idea that for a pilot, a distance judgment is intimately associated with the function of landing. This reflects the ability to use optical invariants to see the world in terms of the capabilities of the aircraft (e.g., reachability and glide angle). These preliminary findings support the role of embodied simulation mechanisms in visual perception and add important insights into a practical understanding of flight expertise, suggesting sensorimotor mechanisms as potential neuro-markers.

The Automation-by-Expertise-by-Training Interaction

OBJECTIVE

I introduce the automation-by-expertise-by-training interaction in automated systems and discuss its influence on operator performance.

BACKGROUND

Transportation accidents that, across a 30-year interval demonstrated identical automation-related operator errors, suggest a need to reexamine traditional views of automation.

METHOD

I review accident investigation reports, regulator studies, and literature on human computer interaction, expertise, and training and discuss how failing to attend to the interaction of automation, expertise level, and training has enabled operators to commit identical automation-related errors.

RESULTS

Automated systems continue to provide capabilities exceeding operators' need for effective system operation and provide interfaces that can hinder, rather than enhance, operator automation-related situation awareness. Because of limitations in time and resources, training programs do not provide operators the expertise needed to effectively operate these automated systems, requiring them to obtain the expertise ad hoc during system operations. As a result, many do not acquire necessary automation-related system expertise.

CONCLUSION

Integrating automation with expected operator expertise levels, and within training programs that provide operators the necessary automation expertise, can reduce opportunities for automation-related operator errors.

APPLICATION

Research to address the automation-by-expertise-by-training interaction is needed. However, such research must meet challenges inherent to examining realistic sociotechnical system automation features with representative samples of operators, perhaps by using observational and ethnographic research. Research in this domain should improve the integration of design and training and, it is hoped, enhance operator performance.

Instructional Design for Accelerated Macrocognitive Expertise in the Baseball Workplace

The goal of accelerating expertise can leave researchers and trainers in human factors, naturalistic decision making, sport science, and expertise studies concerned about seemingly insufficient application of expert performance theories, findings and methods for training macrocognitive aspects of human performance. Video-occlusion methods perfected by sports expertise researchers have great instructional utility, in some cases offering an effective and inexpensive alternative to high-fidelity simulation. A key problem for instructional designers seems to be that expertise research done in laboratory and field settings doesn't get adequately translated into workplace training. Therefore, this article presents a framework for better linkage of expertise research/training across laboratory, field, and workplace settings. It also uses a case study to trace the development and implementation of a macrocognitive training program in the very challenging workplace of the baseball batters' box. This training, which was embedded for a full season in a college baseball team, targeted the perceptual-cognitive skill of pitch recognition that allows expert batters to circumvent limitations of human reaction time in order to hit a 90 mile-per-hour slider. While baseball batting has few analogous skills outside of sports, the instructional design principles of the training program developed to improve batting have wider applicability and implications. Its core operational principle, supported by information processing models but challenged by ecological models, decouples the perception-action link for targeted part-task training of the perception component, in much the same way that motor components routinely are isolated to leverage instructional efficiencies. After targeted perceptual training, perception and action were recoupled via transfer-appropriate tasks inspired by in situ research tasks. Using NCAA published statistics as performance measures, the cooperating team improved from middling performance to first in their conference in Runs Scored and team Batting Average. This case suggests that, beyond the usual considerations of effectiveness and efficiency, there are four challenges to embedded training in the workplace setting -namely: duration, curriculum, limited resources, and buy in. In the case reported here, and potentially in many domains beyond sports, part-task perceptual-cognitive training can improve targeted macrocognitive skills and thereby improve full-skill performance.

An evaluation of the effects of the occupational safety and health inspectors' supervision in workplaces

The aim of this study was to investigate the effects of the administrational supervision of occupational safety and health (OSH) inspectors in the workplaces that they inspect. In the questionnaire survey, the respondents were OSH managers and workers' OSH representatives in the chemical companies' plants. For workers, the aggregated variable 'General OSH enforcement operations' was positively related to the aggregated variable 'Effects of the current OSH inspection'. In contrast, for the OSH managers, the corresponding data did not reveal any correlation. The professional competence in carrying out the OSH inspections was positively related to the different variables of the aggregated variable 'Effects of the current OSH inspection (Outputs)'. In contrast, for the workers' OSH representatives, the analysis failed to confirm this relationship. The present findings are that OSH managers and workers' OSH representatives desired that the OSH inspector should follow up more effectively that OSH management systems are being implemented in practice. They should provide more advice to workplaces in order to help them to exceed the minimum level laid down by law. Workers' OSH representatives expressed clearly that the OSH inspector should more often impose binding obligations. The professional skills of the OSH inspectors need to correspond better to the needs of today's working life and to be able to react to system changes in the workplaces. The OSH enforcement approach should become more uniform and be harmonized between the individual OSH inspectors and the OSH agencies. The development of the quality of OSH inspections and the professional competence of inspectors are important aspects from the point of the effectiveness.

Mental workload as a key factor in clinical decision making

The decision making process is central to the practice of a clinician and has traditionally been described in terms of the hypothetico-deductive model. More recently, models adapted from cognitive psychology, such as the dual process and script theories have proved useful in explaining patterns of practice not consistent with purely cognitive based practice. The purpose of this paper is to introduce the concept of mental workload as a key determinant of the type of cognitive processing used by clinicians. Published research appears to be consistent with 'schemata' based cognition as the principle mode of working for those engaged in complex tasks under time pressure. Although conscious processing of factual data is also used, it may be the primary mode of cognition only in situations where time pressure is not a factor. Further research on the decision making process should be based on outcomes which are not dependant on conscious recall of past actions or events and include a measure of mental workload. This further appears to support the concept of the patient, within the clinical environment, as the most effective learning resource.

Performance of experienced versus less experienced paramedics in managing challenging scenarios: a cognitive task analysis study

STUDY OBJECTIVE

Out-of-hospital care is becoming more complex, thus placing greater reliance on the cognitive abilities of paramedics to manage difficult situations. In adapting to the challenges in their work, paramedics develop expertise. We study the cognitive strategies used by expert paramedics to contribute to understanding how paramedics and the EMS system can adapt to new challenges.

METHODS

We conducted a "staged-world" cognitive task analysis to explore paramedics' handling of cognitive challenges related to sense-making and to resource and task management. A mixed-fidelity simulation was used to present paramedics with 2 challenging scenarios: a pulmonary embolism initially presenting as a myocardial infarction and a 2-person shooting with limited resources available.

RESULTS

Participants were 10 paramedics, 6 more experienced and 4 less experienced. Analysis involved comparing the performance of the 2 groups to identify strategies associated with expertise. The more experienced paramedics made more assessments, explored a wider variety of presumptive diagnoses, and identified the pulmonary embolism earlier. They switched attention between the 2 shooting victims more, used their emergency medical technician-basic level partners more, and provided more advanced level care for both patients. Their patients arrived at the emergency department more prepared for specialized emergency care.

CONCLUSION

Our findings correspond to general cognitive attributes of expertise: greater cue gathering and inferential reasoning, and more functional and strategic thinking. These results suggest potential areas and methods to facilitate development of expertise, as well as ways to better support use of expertise. Future studies should expand on these findings through larger sample sizes and more complex scenarios.

Practice and carryover effects when using small interaction devices

The use of interaction devices in modern work often challenges the human motor system, especially when these devices introduce unfamiliar transformations to the user. In this paper we evaluated expert performance and skill differences between experts and novices when using small motion- and force-controlled interaction devices (touchpad and mini-joystick) in an applied text-editing task. Firstly, experts performed better with their familiar input device than with an unfamiliar one. Particularly touchpad experts operating the unfamiliar mini-joystick showed highly asymmetric carryover costs. Results showed that the efficient performance of experts depended on domain-specific skills, which were not transferable. Secondly, with considerable practice (more than observed for simple and short tasks) novices were brought up to higher levels of performance. The motion-transformation between hand and cursor action was easier in understanding and application than the force-transformation. Thus, the touchpad was used more efficiently than the mini-joystick. In conclusion, practice effects found so far are considerably underestimated when it comes to an applied task. The results give reason to develop and implement skill-sensitive training procedures, since the acquisition of domain-specific skills is critical for expert performance. As a consequence, training procedures might be essential for complex applications and/or unfamiliar device transformations.

Judgment and Decision Making by Individuals and Teams: Issues, Models, and Applications

Consistent with technological advances, the role of the operator in many human factors domains has evolved from one characterized primarily by sensory and motor skills to one characterized primarily by cognitive skills and decision making. Decision making is a primary component in problem solving, human-automation interaction, response to alarms and warnings, and error mitigation. In this chapter we discuss decision making in terms of both front-end judgment processes (e.g., attending to and evaluating the significance of cues and information, formulating a diagnosis, or assessing the situation) and back-end decision processes (e.g., retrieving a course of action, weighing one's options, or mentally simulating a possible response). Two important metatheories—correspondence (empirical accuracy) and coherence (rationality and consistency)—provide ways to assess the goodness of each phase (e.g., Hammond, 1996, 2000; Mosier, 2009). We present several models of decision making, including Brunswik's lens model, naturalistic decision making, and decision ladders, and discuss them in terms of their point of focus and their primary strategies and goals. Next, we turn the discussion to layers in the decision context: individual variables, team decision making, technology, and organizational influences. Last, we focus on applications and lessons learned: investigating, enhancing, designing, and training for decision making. Drawing heavily on sources such as the Human Factors journal, we present recent human factors research exploring these issues, models, and applications.

Guidelines for Training Adaptive Expertise

People operating in today's society frequently encounter complex, dynamic, and ambiguous situations; thus, people must possess mental agility and the ability to adjust quickly yet accurately. Therefore, there is a need to train adaptive expertise to performance successfully. To that end, the purpose of this paper is to provide researchers and practitioners with a review of the literature pertinent to developing training targeting adaptive expertise. This review is presented in the form of guidelines for designing and implementing training interventions specifically focusing on adaptive expertise.