Quantifying the costs of interruption during diagnostic radiology interpretation using mobile eye-tracking glasses

Research on re-searching: interrupted foraging is not disrupted foraging

In classic visual search, observers typically search for the presence of a target in a scene or display. In foraging tasks, there may be multiple targets in the same display (or "patch"). Observers typically search for and collect these target items in one patch until they decide to leave that patch and move to the next one. This is a highly rule-governed behavior. The current study investigated whether these rules are disrupted when the foraging is interrupted in various manners. In Experiment 1, the foraging was briefly interrupted and then resumed in the same patch. In Experiments 2 and 3, the foraging in each patch either ended voluntarily or compulsorily after a fixed amount of time. In these cases, foraging resumed in a patch only after all patches were visited. Overall, the rules of foraging remained largely intact, though Experiment 2 shows that foraging rules can be overridden by the demand characteristics of the task. The results show that participants tended to perform approximately consistently despite interruptions. The results suggest that foraging behavior in a relatively simple foraging environment is resilient and not easily disrupted by interruption.

Interruption in visual search: a systematic review

Visual search, the process of trying to find a target presented among distractors, is a much-studied cognitive task. Less well-studied is the condition in which the search task is interrupted before the target is found. The consequences of such interruptions in visual search have been investigated across various disciplines, which has resulted in diverse and at times contradictory findings. The aim of this systematic review is to provide a more cohesive understanding of the effects of interruptions in visual search. For this purpose, we identified 28 studies that met our inclusion criteria. To facilitate a more organized and comprehensive analysis, we grouped the studies based on three dimensions: the search environment, the interruption aftermath, and the type of the interrupting event. While interruptions in visual search are variable and manifest differently across studies, our review provides a foundational scheme for a more cohesive understanding of the subject. This categorization serves as a starting point for exploring potential future directions, which we delineate in our conclusions.

Reading Room Interruptions are Less Disruptive When Using Asynchronous Communication Methods

Radiologist interruptions, though often necessary, can be disruptive. Prior literature has shown interruptions to be frequent, occurring during cases, and predominantly through synchronous communication methods such as phone or in person causing significant disengagement from the study being read. Asynchronous communication methods are now more widely available in hospital systems such as ours. Considering the increasing use of asynchronous communication methods, we conducted an observational study to understand the evolving nature of radiology interruptions. We hypothesize that compared to interruptions occurring through synchronous methods, interruptions via asynchronous methods reduce the disruptive nature of interruptions by occurring between cases, being shorter, and less severe. During standard weekday hours, 30 radiologists (14 attendings, 12 residents, and 4 fellows) were directly observed for approximately 90-min sessions across three different reading rooms (body, neuroradiology, general). The frequency of interruptions was documented including characteristics such as timing, severity, method, and length. Two hundred twenty-five interruptions (43 Teams, 47 phone, 89 in-person, 46 other) occurred, averaging 2 min and 5 s with 5.2 interruptions per hour. Microsoft Teams interruptions averaged 1 min 12 s with only 60.5% during cases. In-person interruptions averaged 2 min 12 s with 82% during cases. Phone interruptions averaged 2 min and 48 s with 97.9% during cases. A substantial portion of reading room interruptions occur via predominantly asynchronous communication tools, a new development compared to prior literature. Interruptions via predominantly asynchronous communications tools are shorter and less likely to occur during cases. In our practice, we are developing tools and mechanisms to promote asynchronous communication to harness these benefits.

Active visual search in naturalistic environments reflects individual differences in classic visual search performance

Visual search is a ubiquitous activity in real-world environments. Yet, traditionally, visual search is investigated in tightly controlled paradigms, where head-restricted participants locate a minimalistic target in a cluttered array that is presented on a computer screen. Do traditional visual search tasks predict performance in naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality technology to test the degree to which classic and naturalistic search are limited by a common factor, set size, and the degree to which individual differences in classic search behavior predict naturalistic search behavior in a large sample of individuals (N = 75). In a naturalistic search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic search task, participants searched for a target within a simple array of colored letters using only eye-movements. In each task, we found that participants' search performance was impacted by increases in set size-the number of items in the visual display. Critically, we observed that participants' efficiency in classic search tasks-the degree to which set size slowed performance-indeed predicted efficiency in real-world scenes. These results demonstrate that classic, computer-based visual search tasks are excellent models of active, real-world search behavior.

A global and local perspective of interruption frequency in a visual search task

We investigated the impact of frequency of interruptions in a simulated medical visual search task. Participants (N = 150) performed the visual search task during which they were interrupted by a number-classification task in 25, 50, or 75% of all trials, respectively, reflecting the frequency conditions (i.e., low, mid, high). Target presence (i.e., present vs. absent) and interruption (i.e., uninterrupted vs. interrupted) were varied within-subjects, and interruption frequency was varied between-subjects. Globally, on a frequency condition level, participants in the low frequency condition had longer mean response times (RT) for the primary visual search task than in the high condition, but there were no other performance differences between the three frequency conditions. Locally, on the level of specific interruption effects, accuracy decreased directly after interruptions for target present but not for target absent trials. Furthermore, interruptions caused significant interruption costs, reflected in slower overall RTs in interrupted than in uninterrupted trials. The combined findings show that especially for critical visual search tasks as in the medical field, interruptions—regardless of frequency—should be avoided.

Performance of Pediatric Neuroradiologists Working from Home during a Pandemic at a Quaternary Pediatric Academic Hospital

BACKGROUND AND PURPOSE

As a result of the coronavirus disease 2019 (COVID-19) pandemic, many radiology departments shifted to working a portion of clinical assignments from home. To determine the effect of working from home on performance, productivity, quality, and safety, we evaluated turnaround time, volume of studies, and error rates on rotations worked from home compared with in the hospital.

MATERIALS AND METHODS

The number of studies interpreted per day for each neuroradiologist, turnaround times, and error rates reported to peer learning was identified from April 1, 2020, through September 30, 2020. For each neuroradiologist, mean turnaround times and volumes per day at home versus in the hospital were compared. Similar comparison was performed for STAT studies.

RESULTS

During the time period, 2597 CTs (1897 at home, 700 in the hospital) and 3685 MRIs (2601 at home, 1084 in the hospital) were read. By individual neuroradiologists, 57% (4/7) had shorter turnaround time at home and 57% (4/7) demonstrated an increase in the mean number of studies per day read at home. No statistically significant difference was noted in the neuroradiologists' performance while reading STAT studies. Reported error rates were not found to be higher at home, with statistically significantly lower rates when working at home (P = .018).

CONCLUSIONS

Variable productivity and performance of neuroradiologists when working from home versus in the hospital were found, being 57% faster and/or more productive while working at home without an increase in error rates. The decision to work at home versus in the hospital may best be based on local factors, balancing the variability among individual neuroradiologist's and the institution's needs, recognizing that working from home is not a one-size-fits-all phenomenon but requires adaptability for successful implementation.

Characteristics of expert search behavior in volumetric medical image interpretation

Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.

Direct Access and Skill Mix Can Reduce Telephone Interruptions and Imaging Wait Times: Improving Radiology Service Effectiveness, Safety and Sustainability

Unnecessary telephone calls to reporting radiologists impede organizations' workflow and may be associated with a higher chance of errors in reports. We conducted a prospective study in two cycles, which identified vetting plain CT heads as the most common reason for these calls and vetting CT urinary tracts (KUB) was also frequent. Clear vetting and protocolling guidelines exist for both of these scans, which do not routinely require discussion with a radiologist. Therefore, our approach was to create new flow diagrams to allow radiographers to directly accept routine requests for plain CT head and CT KUB scans in- and out-of-hours. After this intervention, incoming calls to radiology for vetting CT heads decreased by 30% and for vetting CT KUBs by 100%. The average wait time between CT head request and scan completion was reduced by 40%. The number of CT head and CT KUB scans performed remained stable. In future, maximizing the benefit of direct access in-patient imaging pathways will rely on effective and sustained communication of the protocols to the junior clinical staff rotating through the organization, as they were responsible for requesting the vast majority of tests.

Eye Tracking for Deep Learning Segmentation Using Convolutional Neural Networks

Deep learning with convolutional neural networks (CNNs) has experienced tremendous growth in multiple healthcare applications and has been shown to have high accuracy in semantic segmentation of medical (e.g., radiology and pathology) images. However, a key barrier in the required training of CNNs is obtaining large-scale and precisely annotated imaging data. We sought to address the lack of annotated data with eye tracking technology. As a proof of principle, our hypothesis was that segmentation masks generated with the help of eye tracking (ET) would be very similar to those rendered by hand annotation (HA). Additionally, our goal was to show that a CNN trained on ET masks would be equivalent to one trained on HA masks, the latter being the current standard approach. Step 1: Screen captures of 19 publicly available radiologic images of assorted structures within various modalities were analyzed. ET and HA masks for all regions of interest (ROIs) were generated from these image datasets. Step 2: Utilizing a similar approach, ET and HA masks for 356 publicly available T1-weighted postcontrast meningioma images were generated. Three hundred six of these image + mask pairs were used to train a CNN with U-net-based architecture. The remaining 50 images were used as the independent test set. Step 1: ET and HA masks for the nonneurological images had an average Dice similarity coefficient (DSC) of 0.86 between each other. Step 2: Meningioma ET and HA masks had an average DSC of 0.85 between each other. After separate training using both approaches, the ET approach performed virtually identically to HA on the test set of 50 images. The former had an area under the curve (AUC) of 0.88, while the latter had AUC of 0.87. ET and HA predictions had trimmed mean DSCs compared to the original HA maps of 0.73 and 0.74, respectively. These trimmed DSCs between ET and HA were found to be statistically equivalent with a p value of 0.015. We have demonstrated that ET can create segmentation masks suitable for deep learning semantic segmentation. Future work will integrate ET to produce masks in a faster, more natural manner that distracts less from typical radiology clinical workflow.

Interruptions in Health Care: Assessing Their Connection With Error and Patient Harm

OBJECTIVE

We address the problem of how researchers investigate the actual or potential causal connection between interruptions and medical errors, and whether interventions might reduce the potential for harm.

BACKGROUND

It is widely assumed that interruptions lead to errors and patient harm. However, many reviewers and authors have commented that there is not strong evidence for a causal connection.

METHOD

We introduce a framework of criteria for assessing how strongly evidence implies causality: the so-called Bradford Hill criteria. We then examine four key "metanarratives" of research into interruptions in health care-applied cognitive psychology, epidemiology, quality improvement, and cognitive systems engineering-and assess how each tradition has addressed the causal connection between interruptions and error.

RESULTS

Outcomes of applying the Bradford Hill criteria are that the applied cognitive psychology and epidemiology metanarratives address the causal connection relatively directly, whereas the quality improvement metanarrative merely assumes causality, and the cognitive systems engineering metanarrative either implicitly or explicitly questions the feasibility of finding a direct causal connection with harm.

CONCLUSION

The Bradford Hill criteria are useful for evaluating the existing literature on the relationship between interruptions in health care, clinical errors, and the potential for patient harm. In the future, more attention is needed to the issue of why interruptions usually do not lead to harm, and the implications for how we approach patient safety.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Eye Movements in Medical Image Perception: A Selective Review of Past, Present and Future

The eye movements of experts, reading medical images, have been studied for many years. Unlike topics such as face perception, medical image perception research needs to cope with substantial, qualitative changes in the stimuli under study due to dramatic advances in medical imaging technology. For example, little is known about how radiologists search through 3D volumes of image data because they simply did not exist when earlier eye tracking studies were performed. Moreover, improvements in the affordability and portability of modern eye trackers make other, new studies practical. Here, we review some uses of eye movements in the study of medical image perception with an emphasis on newer work. We ask how basic research on scene perception relates to studies of medical 'scenes' and we discuss how tracking experts' eyes may provide useful insights for medical education and screening efficiency.