The Impact of Interruptions on Chest Radiograph Interpretation: Effects on Reading Time and Accuracy

A Novel Radiology Communication Tool to Reduce Workflow Interruptions: Clinical Evaluation of RadConnect

Despite the importance of communication, radiology departments often depend on communication tools that were not created for the unique needs of imaging workflows, leading to frequent radiologist interruptions. The objective of this study was test the hypothesis that a novel asynchronous communication tool for the imaging workflow (RadConnect) reduces the daily average number of synchronous (in-person, telephone) communication requests for radiologists. We conducted a before-after study. Before adoption of RadConnect, technologists used three conventional communication methods to consult radiologists (in-person, telephone, general-purpose enterprise chat (GPEC)). After adoption, participants used RadConnect as a fourth method. Technologists manually recorded every radiologist consult request related to neuro and thorax CT scans in the 40 days before and 40 days after RadConnect adoption. Telephone traffic volume to section beepers was obtained from the hospital telephone system for the same period. The value and usability experiences were collected through an electronic survey and structured interviews. RadConnect adoption resulted in 53% reduction of synchronous (in-person, telephone) consult requests: from 6.1 ± 4.2 per day to 2.9 ± 2.9 (P < 0.001). There was 77% decrease (P < 0.001) in telephone volume to the neuro and thorax beepers, while no significant volume change was noted to the abdomen beeper (control group). Survey responses (46% response rate) and interviews confirmed the positive impact of RadConnect on interruptions. RadConnect significantly reduced radiologists' telephone interruptions. Study participants valued the role-based interaction and prioritized worklist overview in the survey and interviews. Findings from this study will contribute to a more focused work environment.

Interruption in visual search: a systematic review

Visual search, the process of trying to find a target presented among distractors, is a much-studied cognitive task. Less well-studied is the condition in which the search task is interrupted before the target is found. The consequences of such interruptions in visual search have been investigated across various disciplines, which has resulted in diverse and at times contradictory findings. The aim of this systematic review is to provide a more cohesive understanding of the effects of interruptions in visual search. For this purpose, we identified 28 studies that met our inclusion criteria. To facilitate a more organized and comprehensive analysis, we grouped the studies based on three dimensions: the search environment, the interruption aftermath, and the type of the interrupting event. While interruptions in visual search are variable and manifest differently across studies, our review provides a foundational scheme for a more cohesive understanding of the subject. This categorization serves as a starting point for exploring potential future directions, which we delineate in our conclusions.

Implementation of an Institution-Wide Rules-Based Automated CT Protocoling System

BACKGROUND. Examination protocoling is a noninterpretive task that increases radiologists' workload and can cause workflow inefficiencies. OBJECTIVE. The purpose of this study was to evaluate effects of an automated CT protocoling system on examination process times and protocol error rates. METHODS. This retrospective study included 317,597 CT examinations (mean age, 61.8 ± 18.1 [SD] years; male, 161,125; female, 156,447; unspecified sex, 25) from July 2020 to June 2022. A rules-based automated protocoling system was implemented institution-wide; the system evaluated all CT orders in the EHR and assigned a protocol or directed the order for manual radiologist protocoling. The study period comprised pilot (July 2020 to December 2020), implementation (January 2021 to December 2021), and postimplementation (January 2022 to June 2022) phases. Proportions of automatically protocoled examinations were summarized. Process times were recorded. Protocol error rates were assessed by counts of quality improvement (QI) reports and examination recalls and comparison with retrospectively assigned protocols in 450 randomly selected examinations. RESULTS. Frequency of automatic protocoling was 19,366/70,780 (27.4%), 68,875/163,068 (42.2%), and 54,045/83,749 (64.5%) in pilot, implementation, and postimplementation phases, respectively (p < .001). Mean (± SD) times from order entry to protocol assignment for automatically and manually protocoled examinations for emergency department examinations were 0.2 ± 18.2 and 2.1 ± 69.7 hours, respectively; mean inpatient examination times were 0.5 ± 50.0 and 3.5 ± 105.5 hours; and mean outpatient examination times were 361.7 ± 1165.5 and 1289.9 ± 2050.9 hours (all p < .001). Mean (± SD) times from order entry to examination completion for automatically and manually protocoled examinations for emergency department examinations were 2.6 ± 38.6 and 4.2 ± 73.0 hours, respectively (p < .001); for inpatient examinations were 6.3 ± 74.6 and 8.7 ± 109.3 hours (p = .001); and for outpatient examinations were 1367.2 ± 1795.8 and 1471.8 ± 2118.3 hours (p < .001). In the three phases, there were three, 19, and 25 QI reports and zero, one, and three recalls, respectively, for automatically protocoled examinations, versus nine, 19, and five QI reports and one, seven, and zero recalls for manually protocoled examinations. Retrospectively assigned protocols were concordant with 212/214 (99.1%) of automatically protocoled versus 233/236 (98.7%) of manually protocoled examinations. CONCLUSION. The automated protocoling system substantially reduced radiologists' protocoling workload and decreased times from order entry to protocol assignment and examination completion; protocol errors and recalls were infrequent. CLINICAL IMPACT. The system represents a solution for reducing radiologists' time spent performing noninterpretive tasks and improving care efficiency.

Reading Room Interruptions are Less Disruptive When Using Asynchronous Communication Methods

Radiologist interruptions, though often necessary, can be disruptive. Prior literature has shown interruptions to be frequent, occurring during cases, and predominantly through synchronous communication methods such as phone or in person causing significant disengagement from the study being read. Asynchronous communication methods are now more widely available in hospital systems such as ours. Considering the increasing use of asynchronous communication methods, we conducted an observational study to understand the evolving nature of radiology interruptions. We hypothesize that compared to interruptions occurring through synchronous methods, interruptions via asynchronous methods reduce the disruptive nature of interruptions by occurring between cases, being shorter, and less severe. During standard weekday hours, 30 radiologists (14 attendings, 12 residents, and 4 fellows) were directly observed for approximately 90-min sessions across three different reading rooms (body, neuroradiology, general). The frequency of interruptions was documented including characteristics such as timing, severity, method, and length. Two hundred twenty-five interruptions (43 Teams, 47 phone, 89 in-person, 46 other) occurred, averaging 2 min and 5 s with 5.2 interruptions per hour. Microsoft Teams interruptions averaged 1 min 12 s with only 60.5% during cases. In-person interruptions averaged 2 min 12 s with 82% during cases. Phone interruptions averaged 2 min and 48 s with 97.9% during cases. A substantial portion of reading room interruptions occur via predominantly asynchronous communication tools, a new development compared to prior literature. Interruptions via predominantly asynchronous communications tools are shorter and less likely to occur during cases. In our practice, we are developing tools and mechanisms to promote asynchronous communication to harness these benefits.

Design and Perceived Value of a Novel Solution for Asynchronous Communication in Radiology

RATIONALE AND OBJECTIVES

Communication with and within the Radiology Department is typically initiated over phone, face-to-face or general-purpose chat, causing frequent interruptions, additional mental workload, workflow inefficiencies and diagnostic errors. We developed and evaluated a new communication solution that aims to reduce avoidable interruptions caused by technologist-radiologist communication.

MATERIALS AND METHODS

Following an iterative design process with future end users, a scalable web-based software solution, RadConnect, was developed enabling a chat-based communication workflow between a technologist and a radiologist. As a first experimental implementation, technologists can send categorized tickets to a radiology section account. Radiologists receive the tickets in a worklist that is prioritized by urgency. Consented radiologists and technologists performed scripted tasks in 2 hr sessions and completed a structured questionnaire on perceived value and comparison to standard communication modes.

RESULTS

Of 17 participants from three academic European institutes, 65% (11/17) believed they would use RadConnect frequently; 53% (9/17) believed that it reduces phone calls >80%; and 88% (15/17) believed it adds value compared to general-purpose enterprise chat applications.

DISCUSSION

Participants recognized the value of RadConnect especially its categorized tickets, prioritized worklist and role-based interaction model. Inter-institute differences in perceived value of RadConnect may have been caused by technologist-radiologist proximity and communication alternatives in the institutions.

CONCLUSION

Chat-based role-based communication might be a viable mode of communication between technologists and radiologists to reduce avoidable interruptions. Tailoring the chat solution to the needs of and tightly integrated with the radiology workflow is valued by future end users after exposure to the tool in a simulated environment.

Implementation of Medical Students as Radiology Reading Room Coordinators

OBJECTIVE

Effort has been made to minimize the burden of non-interpretive tasks (NITs), in particular by hiring and training non-radiologist support staff as reading room coordinators (RRCs). Our medical center recruited and trained senior medical students from our affiliated school of medicine to work alongside on-call radiology residents as RRCs.

METHODS

A 12-month Malpractice Carrier monetary grant was acquired to fund medical students at with the aim to reduce malpractice risk. After the first year, residents were surveyed regarding the impact of the RRCs on perceived on-call efficiency and morale. Furthermore, report turnaround times (TAT) on call shifts that were and were not accompanied by a RRC were compared.

RESULTS

89 % of residents strongly agreed that the RRC improved workflow efficiency, decreased distractions, and felt less stressed during the call shift when the RRC was on duty. 78 % strongly agreed to be more likely to contact a referring clinician when the RRC was able to help coordinate. The mean TAT in the presence of a RRC was 36.8 min, and the mean TAT in the absence of a RRC was 36.9 min DISCUSSION: After hiring medical students to assist on-call radiology residents with noninterpretive tasks, residents reported subjective indicators of program success, but average report turnaround time was unaffected. Nevertheless, we predict that this type of program will continue to grow among academic radiology departments, though additional research is required to evaluate national trends and impacts on radiologist productivity and well-being.

Decreasing Workflow and Educational Interruptions in the Reading Room: Working Smarter, Not Harder

OBJECTIVE

Disruptions in image interpretation lead to interrupted education and inefficiency, and ultimately delay patient care. In the academic reading room, time can often be spent rerouting phone calls. The objective of this study was to evaluate resident perception of current workflow, decrease interruptions, and improve patient care and resident education by implementing a cost-effective automated centralized phone tree.

MATERIALS AND METHODS

Phone call records were obtained between January 25 and February 23, 2021 and May 3 and June 30, 2021 prior to implementation of an automated centralized phone tree within the Emergency Radiology reading room. Calls during weekday business hours were evaluated. Postimplementation phone records were obtained over 4 weeks (August 20-September 16, 2021). Residents on rotation were surveyed prior to and after phone tree implementation regarding the qualitative impact.

RESULTS

The total number of phone calls decreased after phone tree implementation to 8 calls over a 19-day period from 100-200 phone calls over a 20-22 day period. There is a statistically significant difference (p-value < 0.017) in the median number of phone calls postimplementation for all compared preimplementation time points. Resident surveys also demonstrate a statistically significant improvement (p-value < 0.05) in the evaluated metrics.

CONCLUSIONS

Data demonstrate a quantitative decrease in the number of calls arriving at the Emergency Radiology reading room as well as qualitative improvements in resident workplace satisfaction, feelings of burnout, and burden of interruptions. These data suggest that a self-directed triage system (eg, phone tree) could provide a cost-effective and simple means of decreasing interruptions.

Work overload and diagnostic errors in radiology

PURPOSE

To determine the association between workload and diagnostic errors on clinical CT scans.

METHOD

This retrospective study was performed at a tertiary care center and covered the period from January 2020 to March 2023. All clinical CT scans that contained an addendum describing a perceptual error (i.e. failure to detect an important abnormality) in the original report that was issued on office days between 7.30 a.m. and 18.00 p.m., were included. The workload of the involved radiologist on the day of the diagnostic error was calculated in terms of relative value units, and normalized for the known average daily production of each individual radiologist (workloadnormalized). A workloadnormalized of less than 100% indicates relative work underload, while a workloadnormalized of > 100% indicates relative work overload in terms of reported examinations on an individual radiologist's basis.

RESULTS

A total of 49 diagnostic errors were included. Top-five locations of diagnostic errors were lung (n = 8), bone (n = 8), lymph nodes (n = 5), peritoneum (n = 5), and liver (n = 4). Workloadnormalized on the days the diagnostic errors were made was on average 121% (95% confidence interval: 106% to 136%), which was significantly higher than 100% (P = 0.008). There was no significant upward monotonic trend in diagnostic errors over the course of the day (Mann-Kendall tau of 0.005, P = 1.000), and there were no other notable temporal trends either.

CONCLUSIONS

Radiologists appear to have a relative work overload when they make a diagnostic error on CT. Diagnostic errors occurred throughout the entire day, without any increase towards the end of the day.

A global and local perspective of interruption frequency in a visual search task

We investigated the impact of frequency of interruptions in a simulated medical visual search task. Participants (N = 150) performed the visual search task during which they were interrupted by a number-classification task in 25, 50, or 75% of all trials, respectively, reflecting the frequency conditions (i.e., low, mid, high). Target presence (i.e., present vs. absent) and interruption (i.e., uninterrupted vs. interrupted) were varied within-subjects, and interruption frequency was varied between-subjects. Globally, on a frequency condition level, participants in the low frequency condition had longer mean response times (RT) for the primary visual search task than in the high condition, but there were no other performance differences between the three frequency conditions. Locally, on the level of specific interruption effects, accuracy decreased directly after interruptions for target present but not for target absent trials. Furthermore, interruptions caused significant interruption costs, reflected in slower overall RTs in interrupted than in uninterrupted trials. The combined findings show that especially for critical visual search tasks as in the medical field, interruptions—regardless of frequency—should be avoided.

Impact of Concurrent Use of Artificial Intelligence Tools on Radiologists Reading Time: A Prospective Feasibility Study

RATIONAL AND OBJECTIVES

This study investigated how an AI tool impacted radiologists reading time for non-contrast chest CT exams.

MATERIALS AND METHODS

An AI tool was implemented into the PACS reading workflow of non-contrast chest CT exams between April and May 2020. The reading time was recorded for one CONSULTANT RADIOLOGIST and one RADIOLOGY RESIDENT by an external observer. After each case radiologists answered questions regarding additional findings and perceived case overview. Reading times were recorded for 25 cases without and 20 cases with AI tool assistance for each reader. Differences in reading time with and without the AI tool were assessed using Welch's t-test for non-inferiority with non-inferiority limits defined as 100 seconds for the consultant and 200 seconds for the resident.

RESULTS

The mean reading time for the radiology resident was not significantly affected by the AI tool (without AI 370s vs with AI 437s; +67s 95% CI -28s to +163s, p = 0.16). The reading time for the radiology consultant was also not significantly affected by the AI tool (without AI 366s vs with AI 380s; +13s (95% CI - -57s to 84s, p = 0.70). The AI tool led to additional actionable findings in 5/40 (12.5%) studies and better overview in 18/20 (90%) of studies for the resident.

CONCLUSION

A PACS based implementation of an AI tool for concurrent reading of chest CT exams did not increase reading time with additional actionable findings made as well as a perceived better case overview for the radiology resident.

Workflow Interruptions and Effect on Study Interpretation Efficiency

BACKGROUND

Interruptions have been shown to adversely impact efficiency, accuracy, and patient safety.

OBJECTIVE

To analyze the frequency and types of interruptions and effect on report interpretation efficiency.

MATERIALS AND METHODS

A business process improvement team was consulted to make detailed recordings of the activities of the radiologists. Activities were categorized as interpreting studies, active interruptions initiated by the radiologist, and passive interruptions initiated by an external source.

RESULTS

Thirteen board-certified, pediatric radiologists were observed for 61 hours. Radiologists spent 52% of their time interpreting studies, 29% on active interruptions, and 18% on passive interruptions. Approximately 50% of non-interpretive time involved in-person conversations or consults and 16% involved phone calls of which 67% were incoming. The longest time period without an interruption was 20 minutes. 85% of the time, an interruption came within 3 minutes of beginning an interpretation and lasted 1 minute or less 70% of the time. Interruptions increased the time a radiologist needed to read a study by 1 minute for radiographs, 2 minutes for ultrasounds, 6 minutes for CTs, and 10 minutes for magnetic resonance imaging.

CONCLUSION

Total interruption time nearly equaled the total time interpreting studies for radiologists, and interruptions decreased efficiency and increased report interpretation times for all modalities studied. This study highlights the type and extent of interruptions in radiology and examines the effect on report interpretation times. With the frequency of interruptions and impact on efficiency, there is a need to dedicate resources to manage the radiologist workflow. Strategic interventions may ultimately improve outcomes, efficiency, and the overall work environment.

Direct Access and Skill Mix Can Reduce Telephone Interruptions and Imaging Wait Times: Improving Radiology Service Effectiveness, Safety and Sustainability

Unnecessary telephone calls to reporting radiologists impede organizations' workflow and may be associated with a higher chance of errors in reports. We conducted a prospective study in two cycles, which identified vetting plain CT heads as the most common reason for these calls and vetting CT urinary tracts (KUB) was also frequent. Clear vetting and protocolling guidelines exist for both of these scans, which do not routinely require discussion with a radiologist. Therefore, our approach was to create new flow diagrams to allow radiographers to directly accept routine requests for plain CT head and CT KUB scans in- and out-of-hours. After this intervention, incoming calls to radiology for vetting CT heads decreased by 30% and for vetting CT KUBs by 100%. The average wait time between CT head request and scan completion was reduced by 40%. The number of CT head and CT KUB scans performed remained stable. In future, maximizing the benefit of direct access in-patient imaging pathways will rely on effective and sustained communication of the protocols to the junior clinical staff rotating through the organization, as they were responsible for requesting the vast majority of tests.

The Case for User-Centered Artificial Intelligence in Radiology

Past technology transition successes and failures have demonstrated the importance of user-centered design and the science of human factors; these approaches will be critical to the success of artificial intelligence in radiology.

Effects of time of day on radiological interpretation

Accurate interpretation of radiological images involves a complex visual search that relies on several cognitive processes, including selective attention, working memory, and decision-making. Patient outcomes often depend on the accuracy of image interpretations, and yet research has revealed that conclusions vary significantly from one radiologist to another. A myriad of factors has been shown to contribute to the likelihood of interpretative errors and discrepancies, including the radiologist's level of experience and fatigue, and these factors are well reported elsewhere; however, a potentially important factor that has been given little previous consideration is how radiologists' interpretations might be impacted by the time of day at which the reading takes place, a factor that other disciplines have shown to be a determinant of competency. The available literature shows that while the time of day is known to significantly impact some cognitive functions that likely relate to reading competence, including selective visual attention and visual working memory, little is known about the impact of the time of day on radiology interpretation performance. This review explores the evidence regarding the relationship between time of day and performance, with a particular emphasis on radiological activities.

Multidisciplinary Imaging Review Conference Improves Neuro-oncology Radiation Treatment Planning and Follow-up

Purpose: To review the impact of a weekly multidisciplinary neuroradiology imaging review on the management of patients undergoing radiotherapy.

Methods: A prospective study of the management of 118 patients (30=head and neck, 40=skull base, central nervous system=48) was conducted over a 12-month period from January 2018 through January 2019. After review of each patient’s history and relevant imaging, a radiation oncologist completed a form detailing the changes that were made in diagnosis and management. Imaging source (external and internal examinations), availability of outside reports, report timeliness, the value of reports, changes in interpretation, changes in clinical management, and changes in prognosis were documented. Changes in interpretation and management were designated as major or minor depending on the significance of the change. The managing radiation oncologist indicated whether the imaging review conference substituted for a peer-to-peer consultation with a neuroradiologist.

Results: Nearly half (47%) of all patients had a change in interpretation. Of those, 32% of patients had a major change in interpretation, while 14% had a minor change in interpretation. The existence of the multidisciplinary imaging review conference prevented a peer-to-peer consultation (interruption) by the radiation oncologists to the neuroradiologists in 90% of the cases presented. Further analysis was performed.

Conclusion: The involvement of neuroradiologists in a joint radiation oncology imaging review conference resulted in changes in diagnostic imaging interpretation that led to significant changes in management, expected prognosis, and workflow.