Crossing Academic Boundaries for Diagnostic Safety: 10 Complex Challenges and Potential Solutions From Clinical Perspectives and High-Reliability Organizing Principles

OBJECTIVE

We apply the high-reliability organization (HRO) paradigm to the diagnostic process, outlining challenges to enacting HRO principles in diagnosis and offering solutions for how diagnostic process stakeholders can overcome these barriers.

BACKGROUND

Evidence shows that healthcare is starting to organize for higher reliability by employing various principles and practices of HRO. These hold promise for enhancing safer care, but there has been little consideration of the challenges that clinicians and healthcare systems face while enacting HRO principles in the diagnostic process. To effectively deploy the HRO perspective, these barriers must be seriously considered.

METHOD

We review key principles of the HRO paradigm, the diagnostic errors and harms that potentially can be prevented by its enactment, the challenges that clinicians and healthcare systems face in executing various principles and practices, and possible solutions that clinicians and organizational leaders can take to overcome these challenges and barriers.

RESULTS

Our analyses reveal multiple challenges including the inherent diagnostic uncertainty; the lack of diagnosis-focused performance feedback; the fact that diagnosis is often a solo, rather than team, activity; the tendency to simplify the diagnostic process; and professional and institutional status hierarchies. But these challenges are not insurmountable-there are strategies and solutions available to overcome them.

CONCLUSION

The HRO lens offers some important ideas for how the safety of the diagnostic process can be improved.

APPLICATION

The ideas proposed here can be enacted by both individual clinicians and healthcare leaders; both are necessary for making systematic progress in enhancing diagnostic performance.

Redundant Automation Monitoring: Four Eyes Don't See More Than Two, if Everyone Turns a Blind Eye

BACKGROUND

In safety-critical and highly automated environments, more than one person typically monitors the system in order to increase reliability.

OBJECTIVE

We investigate whether the anticipated advantage of redundant automation monitoring is lost due to social loafing and whether individual performance feedback can mitigate this effect.

METHOD

In two experiments, participants worked on a multitasking paradigm in which one task was the monitoring and cross-checking of an automation. Participants worked either alone or with a team partner on this task. The redundant group was further subdivided. One subgroup was instructed that only team performance would be evaluated, whereas the other subgroup expected to receive individual performance feedback after the experiment.

RESULTS

Compared to participants working alone, those who worked collectively but did not expect individual feedback performed significantly less cross-checks and found 25% fewer automation failures. Due to this social loafing effect, even the combined team performance did not surpass the performance of participants working alone. However, when participants expected individual performance feedback, their monitoring behavior and failure detection performance was similar to participants working alone and a team advantage became apparent.

CONCLUSION

Social loafing in redundant automation monitoring can negate the expected gain, if individual performance feedback is not provided.

APPLICATION

These findings may motivate safety experts to evaluate whether their implementation of human redundancy is vulnerable to social loafing effects.

Visual Inspection Reliability for Precision Manufactured Parts

OBJECTIVE

Sandia National Laboratories conducted an experiment for the National Nuclear Security Administration to determine the reliability of visual inspection of precision manufactured parts used in nuclear weapons.

BACKGROUND

Visual inspection has been extensively researched since the early 20th century; however, the reliability of visual inspection for nuclear weapons parts has not been addressed. In addition, the efficacy of using inspector confidence ratings to guide multiple inspections in an effort to improve overall performance accuracy is unknown. Further, the workload associated with inspection has not been documented, and newer measures of stress have not been applied.

METHOD

Eighty-two inspectors in the U.S. Nuclear Security Enterprise inspected 140 parts for eight different defects.

RESULTS

Inspectors correctly rejected 85% of defective items and incorrectly rejected 35% of acceptable parts. Use of a phased inspection approach based on inspector confidence ratings was not an effective or efficient technique to improve the overall accuracy of the process. Results did verify that inspection is a workload-intensive task, dominated by mental demand and effort.

CONCLUSION

Hits for Nuclear Security Enterprise inspection were not vastly superior to the industry average of 80%, and they were achieved at the expense of a high scrap rate not typically observed during visual inspection tasks.

APPLICATION

This study provides the first empirical data to address the reliability of visual inspection for precision manufactured parts used in nuclear weapons. Results enhance current understanding of the process of visual inspection and can be applied to improve reliability for precision manufactured parts.

The Role of Trust as a Mediator Between System Characteristics and Response Behaviors

OBJECTIVE

The purpose of the current work was to clarify how subjective trust determines response behavior when interacting with a signaling system.

BACKGROUND

In multiple theoretical frameworks, trust is acknowledged as a prime mediator between system error characteristics and automation dependence. Some researchers have operationally defined trust as the behavior exhibited. Other researchers have suggested that although trust may guide operator responses, trust does not completely determine the behavior.

METHOD

Forty-four participants interacted with a primary flight simulation task and a secondary signaling system task. The signaling system varied in reliability (90%, 60%) and error bias (false alarm, miss prone). Trust was measured halfway through the experimental session to address the criterion of temporal precedence in determining the effect of trust on behavior.

RESULTS

Analyses indicated that trust partially mediated the relationship between reliability and agreement rate. Trust did not mediate the relationship between reliability and reaction time. Trust also did not mediate the relationships between error bias and reaction time or agreement rate. Analyses of variance generally supported specific behavioral and trust hypotheses, indicating that the paradigm employed produced similar effects on response behaviors and subjective estimates of trust observed in other studies.

CONCLUSION

These results indicate that strong assumptions of trust acting as the prime mediator between system error characteristics and response behaviors should be viewed with caution.

APPLICATION

Practitioners should consider assessing factors other than trust to determine potential operator response behaviors, which may be more predictive.