Distraction in diagnostic radiology: How is search through volumetric medical images affected by interruptions?

Research on re-searching: interrupted foraging is not disrupted foraging

In classic visual search, observers typically search for the presence of a target in a scene or display. In foraging tasks, there may be multiple targets in the same display (or "patch"). Observers typically search for and collect these target items in one patch until they decide to leave that patch and move to the next one. This is a highly rule-governed behavior. The current study investigated whether these rules are disrupted when the foraging is interrupted in various manners. In Experiment 1, the foraging was briefly interrupted and then resumed in the same patch. In Experiments 2 and 3, the foraging in each patch either ended voluntarily or compulsorily after a fixed amount of time. In these cases, foraging resumed in a patch only after all patches were visited. Overall, the rules of foraging remained largely intact, though Experiment 2 shows that foraging rules can be overridden by the demand characteristics of the task. The results show that participants tended to perform approximately consistently despite interruptions. The results suggest that foraging behavior in a relatively simple foraging environment is resilient and not easily disrupted by interruption.

Interruption in visual search: a systematic review

Visual search, the process of trying to find a target presented among distractors, is a much-studied cognitive task. Less well-studied is the condition in which the search task is interrupted before the target is found. The consequences of such interruptions in visual search have been investigated across various disciplines, which has resulted in diverse and at times contradictory findings. The aim of this systematic review is to provide a more cohesive understanding of the effects of interruptions in visual search. For this purpose, we identified 28 studies that met our inclusion criteria. To facilitate a more organized and comprehensive analysis, we grouped the studies based on three dimensions: the search environment, the interruption aftermath, and the type of the interrupting event. While interruptions in visual search are variable and manifest differently across studies, our review provides a foundational scheme for a more cohesive understanding of the subject. This categorization serves as a starting point for exploring potential future directions, which we delineate in our conclusions.

Workflow interruptions in an era of instant messaging: A detailed analysis

INTRODUCTION

The complex practice environment and responsibilities incumbent on diagnostic radiologists creates a workflow susceptible to disruption. While interruptions have been shown to contribute to medical errors in the healthcare delivery environment, the exact impact on highly subspecialized services such as diagnostic radiology is less certain. One potential source of workflow disruption is the use of a departmental instant messaging system (Webex), to facilitate communications between radiology faculty, residents, fellows, and technologists. A retrospective review was conducted to quantify the frequency of interruption experienced by our neuroradiology fellows.

MATERIALS AND METHODS

Data logs were gathered comprising all instant messages sent and received within the designated group chats from July 5-December 31, 2021, during weekday shifts staffed by neuroradiology fellows. Interruptions per shift were calculated based on month, week, and day of the week.

RESULTS

14,424 messages were sent across 289 total shifts. The 6 fellows assigned to the main neuroradiology reading room sent 3258 messages and received 10,260 messages from technologists and other staff. There was an average of 50 interruptions per shift when examined by month (range 48-53), and 52 interruptions per shift when examined by day of the week (range 40-60).

CONCLUSION

Neuroradiology fellows experience frequent interruptions from the departmental instant messaging system. These disruptions, when considered in conjunction with other non-interpretative tasks, may have negative implications for workflow efficiency, requiring iterative process improvements when incorporating new technology into the practice environment of diagnostic radiology.

Factors Contributing to Disproportionate Burnout in Women Breast Imaging Radiologists: A Review

Physician burnout continues to increase in prevalence and disproportionately affects women physicians. Breast imaging is a woman-dominated subspeciality, and therefore, worsening burnout among women physicians may have significant repercussions on the future of the breast imaging profession. Systemic and organizational factors have been shown to be the greatest contributors to burnout beyond individual factors. Based on the Mayo Model, we review the evidence regarding the 7 major organizational contributors to physician burnout and their potential disproportionate impacts on women breast radiologists. The major organizational factors discussed are work-life integration, control and flexibility, workload and job demands, efficiency and resources, finding meaning in work, social support and community at work, and organizational culture and values. We also propose potential strategies for institutions and practices to mitigate burnout in women breast imaging radiologists. Many of these strategies could also benefit men breast imaging radiologists, who are at risk for burnout as well.

The impact of different distractions on outdoor visual search and object memory

We investigated whether and how different types of search distractions affect visual search behavior and target memory while participants searched in a real-world environment. They searched either undistracted (control condition), listened to a podcast (auditory distraction), counted down aloud at intervals of three while searching (executive working memory load), or were forced to stop the search on half of the trials (time pressure). In line with findings from laboratory settings, participants searched longer but made fewer errors when the target was absent than when it was present, regardless of distraction condition. Furthermore, compared to the auditory distraction condition, the executive working memory load led to higher error rates (but not longer search times). In a surprise memory test after the end of the search tasks, recognition was better for previously present targets than for absent targets. Again, this was regardless of the previous distraction condition, although significantly fewer targets were remembered by the participants in the executive working memory load condition than by those in the control condition. The findings suggest that executive working memory load, but likely not auditory distraction and time pressure affected visual search performance and target memory in a real-world environment.

A global and local perspective of interruption frequency in a visual search task

We investigated the impact of frequency of interruptions in a simulated medical visual search task. Participants (N = 150) performed the visual search task during which they were interrupted by a number-classification task in 25, 50, or 75% of all trials, respectively, reflecting the frequency conditions (i.e., low, mid, high). Target presence (i.e., present vs. absent) and interruption (i.e., uninterrupted vs. interrupted) were varied within-subjects, and interruption frequency was varied between-subjects. Globally, on a frequency condition level, participants in the low frequency condition had longer mean response times (RT) for the primary visual search task than in the high condition, but there were no other performance differences between the three frequency conditions. Locally, on the level of specific interruption effects, accuracy decreased directly after interruptions for target present but not for target absent trials. Furthermore, interruptions caused significant interruption costs, reflected in slower overall RTs in interrupted than in uninterrupted trials. The combined findings show that especially for critical visual search tasks as in the medical field, interruptions—regardless of frequency—should be avoided.

Workflow Interruptions and Effect on Study Interpretation Efficiency

BACKGROUND

Interruptions have been shown to adversely impact efficiency, accuracy, and patient safety.

OBJECTIVE

To analyze the frequency and types of interruptions and effect on report interpretation efficiency.

MATERIALS AND METHODS

A business process improvement team was consulted to make detailed recordings of the activities of the radiologists. Activities were categorized as interpreting studies, active interruptions initiated by the radiologist, and passive interruptions initiated by an external source.

RESULTS

Thirteen board-certified, pediatric radiologists were observed for 61 hours. Radiologists spent 52% of their time interpreting studies, 29% on active interruptions, and 18% on passive interruptions. Approximately 50% of non-interpretive time involved in-person conversations or consults and 16% involved phone calls of which 67% were incoming. The longest time period without an interruption was 20 minutes. 85% of the time, an interruption came within 3 minutes of beginning an interpretation and lasted 1 minute or less 70% of the time. Interruptions increased the time a radiologist needed to read a study by 1 minute for radiographs, 2 minutes for ultrasounds, 6 minutes for CTs, and 10 minutes for magnetic resonance imaging.

CONCLUSION

Total interruption time nearly equaled the total time interpreting studies for radiologists, and interruptions decreased efficiency and increased report interpretation times for all modalities studied. This study highlights the type and extent of interruptions in radiology and examines the effect on report interpretation times. With the frequency of interruptions and impact on efficiency, there is a need to dedicate resources to manage the radiologist workflow. Strategic interventions may ultimately improve outcomes, efficiency, and the overall work environment.

Mandating Limits on Workload, Duty, and Speed in Radiology

Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.

Burnout: A Mindful Framework for the Radiologist

Burnout, the outcome of prolonged stress or frustration, manifests as both mental and physical fatigue affecting over half of healthcare workers. This article will discuss the etiologies, problems, and potential solutions to burnout related issues that are impacting radiologists. Factors placing radiologists at risk for burnout as well the impact of burnout upon the radiologist, the department, staff, and patients they serve will also be discussed. An emphasis will also be placed upon recognition, solutions, and a collective response to burnout. Readers should be able to perform a self-assessment of their own risk for burnout and understand what can be done to dissolve and prevent burnout amongst their colleagues. In doing so, our hope is that radiologists will develop greater insight, awareness, and ultimately empathy for the unique challenges that others in the radiology community may face.

The performance costs of interruption during visual search are determined by the type of search task

Prior research has shown that interruptions lead to a variety of performance costs. However, these costs are heterogenous and poorly understood. Under some circumstances, interruptions lead to large decreases in accuracy on the primary task, whereas in others task duration increases, but task accuracy is unaffected. Presently, the underlying cause of these costs is unclear. The Memory for Goals model suggests that interruptions interfere with the ability to represent the current goal of the primary task. Here, we test the idea that working memory (WM) may play a critical role in representing the current goal and thus may underlie the observed costs associated with interruption. In two experiments, we utilized laboratory-based visual search tasks, which differed in their WM demands, in order to assess how this difference influenced the observed interruption costs. Interruptions led to more severe performance costs when the target of the search changed on each trial. When the search target was consistent across trials, the cost of interruption was greatly reduced. This suggests that the WM demands associated with the primary task play an important role in determining the performance costs of interruption. Our findings suggest that it is important for research to consider the cognitive processes a task engages in order to predict the nature of the adverse effects of interruption in applied settings such as radiology.

Characteristics of expert search behavior in volumetric medical image interpretation

Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.

Distractions in the cardiac catheterisation laboratory: impact for cardiologists and patient safety

Objective

To understand human factors (HF) contributing to disturbances during invasive cardiac procedures, including frequency and nature of distractions, and assessment of operator workload.

Methods

Single centre prospective observational evaluation of 194 cardiac procedures in three adult cardiac catheterisation laboratories over 6 weeks. A proforma including frequency, nature, magnitude and level of procedural risk at the time of each distraction/interruption was completed for each case. The primary operator completed a National Aeronautical and Space Administration (NASA) task load questionnaire rating mental/physical effort, level of frustration, time-urgency, and overall effort and performance.

Results

264 distractions occurred in 106 (55%) out of 194 procedures observed; 80% were not relevant to the case being undertaken; 14% were urgent including discussions of potential ST-elevation myocardial infarction requiring emergency angioplasty. In procedures where distractions were observed, frequency per case ranged from 1 to 16 (mean 2.5, SD ±2.2); 43 were documented during high-risk stages of the procedure. Operator rating of NASA task load parameters demonstrated higher levels of mental and physical workload and effort during cases in which distractions occurred.

Conclusions

In this first description of HF in adult cardiac catheter laboratories, we found that fewer than half of all procedures were completed without interruption/distraction. The majority were unnecessary and without relation to the case or list. We propose the introduction of a ‘sterile cockpit’ environment within catheter laboratories, as adapted from aviation and used in surgical operating theatres, to minimise non-emergent interruptions and disturbances, to improve operator conditions and overall patient safety.

The cognitive and perceptual processes that affect observer performance in lung cancer detection: a scoping review

INTRODUCTION

Early detection of malignant pulmonary nodules through screening has been shown to reduce lung cancer-related mortality by 20%. However, perceptual and cognitive factors that affect nodule detection are poorly understood. This review examines the cognitive and visual processes of various observers, with a particular focus on radiologists, during lung nodule detection.

METHODS

Four databases (Medline, Embase, Scopus and PubMed) were searched to extract studies on eye-tracking in pulmonary nodule detection. Studies were included if they used eye-tracking to assess the search and detection of lung nodules in computed tomography or 2D radiographic imaging. Data were charted according to identified themes and synthesised using a thematic narrative approach.

RESULTS

The literature search yielded 25 articles and five themes were discovered: 1 - functional visual field and satisfaction of search, 2 - expert search patterns, 3 - error classification through dwell time, 4 - the impact of the viewing environment and 5 - the effect of prevalence expectation on search. Functional visual field reduced to 2.7° in 3D imaging compared to 5° in 2D radiographs. Although greater visual coverage improved nodule detection, incomplete search was not responsible for missed nodules. Most radiological errors during lung nodule detection were decision-making errors (30%-45%). Dwell times associated with false-positive (FP) decisions informed feedback systems to improve diagnosis. Interruptions did not influence diagnostic performance; however, it increased viewing time by 8% and produced a 23.1% search continuation accuracy. Comparative scanning was found to increase the detection of low contrast nodules. Prevalence expectation did not directly affect diagnostic accuracy; however, decision-making time increased by 2.32 seconds with high prevalence expectations.

CONCLUSION

Visual and cognitive factors influence pulmonary nodule detection. Insights gained from eye-tracking can inform advancements in lung screening. Further exploration of eye-tracking in lung screening, particularly with low-dose computed tomography (LDCT), will benefit the future of lung cancer screening.

Accelerated implementation of remote reporting during the COVID-19 pandemic

AIM

To assess, via a survey of UK radiological departments, if the COVID-19 pandemic led to a change in radiological reporting undertaken in a home environment with appropriate IT support.

MATERIALS AND METHODS

All imaging departments in the UK were contacted and asked about the provision of home reporting and IT support before and after the first wave of the pandemic.

RESULTS

One hundred and thirty-seven of the 217 departments contacted replied, producing a response rate of 61%. There was a 147% increase in the provision of remote access viewing and reporting platforms during the pandemic. Although 578 consultants had access to a viewing platform pre-pandemic, this had increased to 1,431 during the course of the first wave.

CONCLUSION

This survey represents work undertaken by UK NHS Trusts in co-ordinating and providing increased home-reporting facilities to UK radiologists during the first wave of this global pandemic. The impact of these facilities has been shown to allow more than just the provision of reporting of both elective and emergency imaging and provides additional flexibility in how UK radiologists can help support and provide services. This is a good start, but there are potential problems that now need to be overcome.

Survey of radiologists and emergency department providers after implementation of a global radiology report categorization system

PURPOSE

Breakdown in communication of important imaging results threatens patient safety and risks malpractice claims. To facilitate closed-loop communication, our institution developed a unique radiology report categorization (RADCAT) system employing automated alert notification systems. This study aimed to understand users' initial experiences with the RADCAT system and obtain feedback.

METHODS

Web-based surveys were distributed to radiologists and emergency department (ED) providers at our hospital system within 1 year of institution-wide RADCAT implementation. Survey designs differed based on clinical setting. Most prompts utilized declarative statements with 5-point agreement Likert scales. Closed-response data was analyzed with descriptive statistics.

RESULTS

Response rates among radiologists and ED providers were 59.4% (63/106) and 38.4% (69/211), respectively. 78.0% (46/59) of radiologists and 60.9% (42/69) of ED providers agreed that RADCAT improves patient care. Of radiologists, 84.1% (53/63) agreed that RADCAT design is intuitive, and 57.6% (34/59) agreed that RADCAT improves efficiency. Of ED providers, 69.6% (48/69) agreed that RADCAT appropriately differentiates urgent and non-urgent findings, and 65.2% (45/69) agreed that auto-population of discharge documents with imaging results containing follow-up recommendations protects them from liability. Only 35.6% (21/59) of radiologists and 21.7% (15/69) of ED providers agreed that RADCAT implementation decreased reading room visits by ordering providers. Open-response feedback showed that some ED providers find RADCAT too complex while some radiologists desire improved transparency regarding imaging study communication status.

CONCLUSION

Since its implementation, RADCAT has been well received among radiologists and ED providers with agreement that it improves patient care and effectively distinguishes and communicates important imaging findings.

The Radiologist's Gaze: Mapping Three-Dimensional Visual Search in Computed Tomography of the Abdomen and Pelvis

A radiologist's search pattern can directly influence patient management. A missed finding is a missed opportunity for intervention. Multiple studies have attempted to describe and quantify search patterns but have mainly focused on chest radiographs and chest CTs. Here, we describe and quantify the visual search patterns of 17 radiologists as they scroll through 6 CTs of the abdomen and pelvis. Search pattern tracings varied among individuals and remained relatively consistent per individual between cases. Attendings and trainees had similar eye metric statistics with respect to time to first fixation (TTFF), number of fixations in the region of interest (ROI), fixation duration in ROI, mean saccadic amplitude, or total number of fixations. Attendings had fewer numbers of fixations per second versus trainees (p < 0.001), suggesting efficiency due to expertise. In those cases that were accurately interpreted, TTFF was shorter (p = 0.04), the number of fixations per second and number of fixations in ROI were higher (p = 0.04, p = 0.02, respectively), and fixation duration in ROI was increased (p = 0.02). We subsequently categorized radiologists as "scanners" or "drillers" by both qualitative and quantitative methods and found no differences in accuracy with most radiologists being categorized as "drillers." This study describes visual search patterns of radiologists in interpretation of CTs of the abdomen and pelvis to better approach future endeavors in determining the effects of manipulations such as fatigue, interruptions, and computer-aided detection.

Measurement of the useful field of view for single slices of different imaging modalities and targets

Purpose: With three-dimensional (3-D) images displayed as stacks of 2-D images, radiologists rely more heavily on vision away from their fixation point to visually process information, guide eye movements, and detect abnormalities. Thus the ability to detect targets away from the fixation point, commonly characterized as the useful field of view (UFOV), becomes critical for these 3-D imaging modalities. We investigate how the UFOV, defined as the eccentricity, in which detection performance degrades to a given probability, varies across imaging modalities and targets. Approach: We measure the detectability of different targets at various distances from gaze locations for single slices of liver computed tomography (CT), 2-D digital mammograms (DM), and single slices of digital breast tomosynthesis (DBT) cases. Observers with varying expertise were instructed to maintain their gaze at a point while a short display of the image was flashed and an eye tracker verified observer's steady fixation. Display times were 200 and 1000 ms for CT images and 500 ms for DM and DBT images. Results: We find variations in the UFOV from 9 to 12 deg for liver CT to as small as 2.5 to 5 deg for calcification clusters in breast images (DM and DBT). We compare our results to those reported in the literature for lung nodules and discuss the differences across methods used to measure the UFOV, their dependence on case selection/task difficulty, viewing conditions, and observer expertise. We propose a complementary measure defined in terms of performance degradation relative to the peak foveal performance (relative-UFOV) to circumvent UFOV's variations with case selection/task difficulty. Conclusion: Our results highlight the variations in the UFOV across imaging modalities, target types, observer expertise, and measurement methods and suggest an additional relative-UFOV measure to more thoroughly characterize the detection performance away from point of fixation.

Evaluating Radiology Result Communication in the Emergency Department

PURPOSE

To assess the pattern of result communication that occurs between radiologists and referring physicians in the emergency department setting.

METHODS

An institutional review board-approved prospective study was performed at a large academic medical center with 24/7 emergency radiology cover. Emergency radiologists logged information regarding all result-reporting communication events that occurred over a 168-hour period.

RESULTS

A total of 286 independent result communication events occurred during the study period, the vast majority of which occurred via telephone (232/286). Emergency radiologists spent 10% of their working time communicating results. Similar amounts of time were spent discussing negative and positive cross-sectional imaging examinations. In a small minority of communication events, additional information was gathered through communication that resulted in a change of interpretation from a normal to an abnormal study.

CONCLUSIONS

Effective and efficient result communication is critical to care delivery in the emergency department setting. Discussion regarding abnormal cases, both in person and over the phone, is encouraged. However, in the emergency setting, time spent on routine direct communication of negative examination results in advance of the final report may lead to increased disruptions, longer turnaround times, and negatively impact patient care. In very few instances, does the additional information gained from the communication event result in a change of interpretation?

Better efficacy in differentiating WHO grade II from III oligodendrogliomas with machine-learning than radiologist's reading from conventional T1 contrast-enhanced and fluid attenuated inversion recovery images

Background

The medical imaging to differentiate World Health Organization (WHO) grade II (ODG2) from III (ODG3) oligodendrogliomas still remains a challenge. We investigated whether combination of machine leaning with radiomics from conventional T1 contrast-enhanced (T1 CE) and fluid attenuated inversion recovery (FLAIR) magnetic resonance imaging (MRI) offered superior efficacy.

Methods

Thirty-six patients with histologically confirmed ODGs underwent T1 CE and 33 of them underwent FLAIR MR examination before any intervention from January 2015 to July 2017 were retrospectively recruited in the current study. The volume of interest (VOI) covering the whole tumor enhancement were manually drawn on the T1 CE and FLAIR slice by slice using ITK-SNAP and a total of 1072 features were extracted from the VOI using 3-D slicer software. Random forest (RF) algorithm was applied to differentiate ODG2 from ODG3 and the efficacy was tested with 5-fold cross validation. The diagnostic efficacy of radiomics-based machine learning and radiologist’s assessment were also compared.

Results

Nineteen ODG2 and 17 ODG3 were included in this study and ODG3 tended to present with prominent necrosis and nodular/ring-like enhancement (P < 0.05). The AUC, ACC, sensitivity, and specificity of radiomics were 0.798, 0.735, 0.672, 0.789 for T1 CE, 0.774, 0.689, 0.700, 0.683 for FLAIR, as well as 0.861, 0.781, 0.778, 0.783 for the combination, respectively. The AUCs of radiologists 1, 2 and 3 were 0.700, 0.687, and 0.714, respectively. The efficacy of machine learning based on radiomics was superior to the radiologists’ assessment.

Conclusions

Machine-learning based on radiomics of T1 CE and FLAIR offered superior efficacy to that of radiologists in differentiating ODG2 from ODG3.

Distracting Objects Induce Early Quitting in Visual Search

Task-irrelevant objects can sometimes capture attention and increase the time it takes an observer to find a target. However, less is known about how these distractors impact visual search strategies. Here, I found that salient distractors reduced rather than increased response times on target-absent trials (Experiment 1; N = 200). Combined with higher error rates on target-present trials, these results indicate that distractors can induce observers to quit search earlier than they otherwise would. These effects were replicated when target prevalence was low (Experiment 2; N = 200) and with different stimuli that elicited shallower search slopes (Experiment 3; N = 75). These results demonstrate that salient distractors can produce at least two consequences in visual search: They can capture attention, and they can cause observers to quit searching early. This novel finding has implications both for understanding visual attention and for examining distraction in real-world domains where targets are often absent, such as medical image screening.

Quantifying disruption of workflow by phone calls to the neuroradiology reading room

Introduction

The purpose of this study was to understand the source and the reason for the phone calls to our neuroradiology suit and to quantify the size of the problem in terms of duration of individual and aggregated calls.

Materials and methods

Observation of the neuroradiology reading room for the entire duration of the working hours over three non-consecutive days was performed, and included telephone calls start time, end time and calls duration for incoming telephone calls. After each phone call the recipients were queried on the details of the phone call; the origin of the call, the reason for the call and the response.

Results

The average total number of minutes (min) spent on the phone each day was 64 min per working day with a total of 39 phone calls per day and 4.4 per hour on average. The trainees answered 71% of the phone calls with additional intervention by attending in 13% of phone calls. The most common source of phone calls was from either the MRI/CT technicians (48%), followed by providers (20%) and returning pages (18%).

Conclusion

Cumulative time spent on the phone by neuroradiologists in the reading room ended up in more than an hour per working day, while trainees were taking the majority of phone calls. Most phone calls originated from technicians, hence, requiring specific solutions to mitigate this kind of interruption.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Eye Movements in Medical Image Perception: A Selective Review of Past, Present and Future

The eye movements of experts, reading medical images, have been studied for many years. Unlike topics such as face perception, medical image perception research needs to cope with substantial, qualitative changes in the stimuli under study due to dramatic advances in medical imaging technology. For example, little is known about how radiologists search through 3D volumes of image data because they simply did not exist when earlier eye tracking studies were performed. Moreover, improvements in the affordability and portability of modern eye trackers make other, new studies practical. Here, we review some uses of eye movements in the study of medical image perception with an emphasis on newer work. We ask how basic research on scene perception relates to studies of medical 'scenes' and we discuss how tracking experts' eyes may provide useful insights for medical education and screening efficiency.

Interruptions and Delivery of Care in the Intensive Care Unit

OBJECTIVE

This study samples interruption frequency in intensive care unit (ICU) settings to assess the relationship between interruptions and common patient hazards.

BACKGROUND

Task interruptions are accident contributors in numerous industries. Recently, studies on health care interruptions and their impact on patient hazards have received attention.

METHOD

Seven ICUs in four hospitals participated in a 24-month study. Experienced ICU nurses directly observed nursing tasks, interruptions, and patient hazards (delays in care, breaks in device task protocols, and patient safety hazards).

RESULTS

During 1,148 hours of observation, 175 nurses performed 74,733 nursing tasks. Interruptions occurred at a rate of 4.95 per hour, and 8.4% of tasks were interrupted. Interruptions originated mostly from humans (65.9%), alarms (24.1%), and others (10%). A total of 774 patient hazards were observed, with a hazard occurring on average every 89 minutes. Relative to noninterrupted tasks, device alarm interrupted nonstructured tasks were associated with increased rates of delays in care and safety hazards (rate ratio [RR] = 3.19). In contrast, rate of delays in care and safety hazards did not increase during human interrupted tasks (RR = 1.13). Rates of protocol nonadherence varied by device type and were highest during artificial airway, medication administration, chest tube, and supplemental oxygen management.

CONCLUSION

Interruptions in the ICU are frequent and contribute to patient hazards, especially when caused by device alarms during nonstructured tasks. Nonadherence to protocols is common and contributed to patient hazards.

APPLICATION

The findings suggest a need for improvement in task and device design to reduce patient hazards.

Detecting insertion, substitution, and deletion errors in radiology reports using neural sequence-to-sequence models

Background

Errors in grammar, spelling, and usage in radiology reports are common. To automatically detect inappropriate insertions, deletions, and substitutions of words in radiology reports, we proposed using a neural sequence-to-sequence (seq2seq) model.

Methods

Head CT and chest radiograph reports from Mount Sinai Hospital (MSH) (n=61,722 and 818,978, respectively), Mount Sinai Queens (MSQ) (n=30,145 and 194,309, respectively) and MIMIC-III (n=32,259 and 54,685) were converted into sentences. Insertions, substitutions, and deletions of words were randomly introduced. Seq2seq models were trained using corrupted sentences as input to predict original uncorrupted sentences. Three models were trained using head CTs from MSH, chest radiographs from MSH, and head CTs from all three collections. Model performance was assessed across different sites and modalities. A sample of original, uncorrupted sentences were manually reviewed for any error in syntax, usage, or spelling to estimate real-world proofreading performance of the algorithm.

Results

Seq2seq detected 90.3% and 88.2% of corrupted sentences with 97.7% and 98.8% specificity in same-site, same-modality test sets for head CTs and chest radiographs, respectively. Manual review of original, uncorrupted same-site same-modality head CT sentences demonstrated seq2seq positive predictive value (PPV) 0.393 (157/400; 95% CI, 0.346-0.441) and negative predictive value (NPV) 0.986 (789/800; 95% CI, 0.976-0.992) for detecting sentences containing real-world errors, with estimated sensitivity of 0.389 (95% CI, 0.267-0.542) and specificity 0.986 (95% CI, 0.985-0.987) over n=86,211 uncorrupted training examples.

Conclusions

Seq2seq models can be highly effective at detecting erroneous insertions, deletions, and substitutions of words in radiology reports. To achieve high performance, these models require site- and modality-specific training examples. Incorporating additional targeted training data could further improve performance in detecting real-world errors in reports.

Distractors in obstetric ultrasound: Do sonographers have safety concerns?

INTRODUCTION

Obstetric sonography is a highly skilled diagnostic medical examination. Pregnant women like to socialise their ultrasound experience with family, introducing distractions for the sonographer. Our objective was to survey ultrasound practitioners to identify concerns regarding interruptions and their opinions about socialisation during the examination.

METHODS

An online questionnaire was disseminated to study the views of Australian and New Zealand obstetric sonographers/sonologists. It was informed by a pilot study of possible distractors with quality and safety concerns and operator opinions regarding family bonding.

RESULTS

The opinions of 393 obstetric sonographers/sonologists informed our results. Distractors with the most negative aspects included disruptive children (93.3%) and mobile phone conversations (84.3%). Most respondents (62%) believed that a distractor only had to be present for 5 min or less to have an impact. Small children were identified by 87.5% of respondents as safety risks to themselves, to the patient and to sonographers. Sonographers were concerned that distractors caused a loss of concentration, interruption to a systematic scanning approach and increased false negatives in screening, missing important diagnoses. Sonographers strongly agreed that obstetric sonography facilitated maternal-fetal bonding, but only 15% thought that siblings bond with the fetus during the scan.

CONCLUSION

Obstetric sonographers in our study are concerned that distractors pose a negative impact on the quality and safety of ultrasound. They also recognise the importance of family bonding. Strategies to bridge the medical and social components of obstetric sonography should be developed to reduce quality and safety threats.

Machine learning: a useful radiological adjunct in determination of a newly diagnosed glioma's grade and IDH status

INTRODUCTION

Machine learning methods have been introduced as a computer aided diagnostic tool, with applications to glioma characterisation on MRI. Such an algorithmic approach may provide a useful adjunct for a rapid and accurate diagnosis of a glioma. The aim of this study is to devise a machine learning algorithm that may be used by radiologists in routine practice to aid diagnosis of both: WHO grade and IDH mutation status in de novo gliomas.

METHODS

To evaluate the status quo, we interrogated the accuracy of neuroradiology reports in relation to WHO grade: grade II 96.49% (95% confidence intervals [CI] 0.88, 0.99); III 36.51% (95% CI 0.24, 0.50); IV 72.9% (95% CI 0.67, 0.78). We derived five MRI parameters from the same diagnostic brain scans, in under two minutes per case, and then supplied these data to a random forest algorithm.

RESULTS

Machine learning resulted in a high level of accuracy in prediction of tumour grade: grade II/III; area under the receiver operating characteristic curve (AUC) = 98%, sensitivity = 0.82, specificity = 0.94; grade II/IV; AUC = 100%, sensitivity = 1.0, specificity = 1.0; grade III/IV; AUC = 97%, sensitivity = 0.83, specificity = 0.97. Furthermore, machine learning also facilitated the discrimination of IDH status: AUC of 88%, sensitivity = 0.81, specificity = 0.77.

CONCLUSIONS

These data demonstrate the ability of machine learning to accurately classify diffuse gliomas by both WHO grade and IDH status from routine MRI alone-without significant image processing, which may facilitate usage as a diagnostic adjunct in clinical practice.

Quantifying the costs of interruption during diagnostic radiology interpretation using mobile eye-tracking glasses

What are the costs and consequences of interruptions during diagnostic radiology? The cognitive psychology literature suggests that interruptions lead to an array of negative consequences that could hurt patient outcomes and lead to lower patient throughput. Meanwhile, observational studies have both noted a strikingly high rate of interruptions and rising number of interruptions faced by radiologists. There is some observational evidence that more interruptions could lead to worse patient outcomes: Balint et al. (2014) found that the shifts with more telephone calls received in the reading room were associated with more discrepant calls. The purpose of the current study was to use an experimental manipulation to precisely quantify the costs of two different types of interruption: telephone interruption and an interpersonal interruption. We found that the first telephone interruption led to a significant increase in time spent on the case, but there was no effect on diagnostic accuracy. Eye-tracking revealed that interruptions strongly influenced where the radiologists looked: they tended to spend more time looking at dictation screens and less on medical images immediately after interruption. Our results demonstrate that while radiologists’ eye movements are reliably influenced by interruptions, the behavioral consequences were relatively mild, suggesting effective compensatory mechanisms.

Errors in Mammography Cannot be Solved Through Technology Alone

Mammography has been the frontline screening tool for breast cancer for decades. However, high error rates in the form of false negatives (FNs) and false positives (FPs) have persisted despite technological improvements. Radiologists still miss between 10% and 30% of cancers while 80% of woman recalled for additional views have normal outcomes, with 40% of biopsied lesions being benign. Research show that the majority of cancers missed is actually visible and looked at, but either go unnoticed or are deemed to be benign. Causal agents for these errors include human related characteristics resulting in contributory search, perception and decision-making behaviours. Technical, patient and lesion factors are also important relating to positioning, compression, patient size, breast density and presence of breast implants as well as the nature and subtype of the cancer itself, where features such as architectural distortion and triple-negative cancers remain challenging to detect on screening. A better understanding of these causal agents as well as the adoption of technological and educational interventions, which audits reader performance and provide immediate perceptual feedback, should help. This paper reviews the current status of our knowledge around error rates in mammography and explores the factors impacting it. It also presents potential solutions for maximizing diagnostic efficacy thus benefiting the millions of women who undergo this procedure each year.