What do radiologists look for? Advances and limitations of perceptual learning in radiologic search

Bridging human and machine intelligence: Reverse-engineering radiologist intentions for clinical trust and adoption

In the rapidly evolving landscape of medical imaging, the integration of artificial intelligence (AI) with clinical expertise offers unprecedented opportunities to enhance diagnostic precision and accuracy. Yet, the "black box" nature of AI models often limits their integration into clinical practice, where transparency and interpretability are important. This paper presents a novel system leveraging the Large Multimodal Model (LMM) to bridge the gap between AI predictions and the cognitive processes of radiologists. This system consists of two core modules, Temporally Grounded Intention Detection (TGID) and Region Extraction (RE). The TGID module predicts the radiologist's intentions by analyzing eye gaze fixation heatmap videos and corresponding radiology reports. Additionally, the RE module extracts regions of interest that align with these intentions, mirroring the radiologist's diagnostic focus. This approach introduces a new task, radiologist intention detection, and is the first application of Dense Video Captioning (DVC) in the medical domain. By making AI systems more interpretable and aligned with radiologist's cognitive processes, this proposed system aims to enhance trust, improve diagnostic accuracy, and support medical education. Additionally, it holds the potential for automated error correction, guiding junior radiologists, and fostering more effective training and feedback mechanisms. This work sets a precedent for future research in AI-driven healthcare, offering a pathway towards transparent, trustworthy, and human-centered AI systems. We evaluated this model using NLG(Natural Language Generation), time-related, and vision-based metrics, demonstrating superior performance in generating temporally grounded intentions on REFLACX and EGD-CXR datasets. This model also demonstrated strong predictive accuracy in overlap scores for medical abnormalities and effective region extraction with high IoU(Intersection over Union), especially in complex cases like cardiomegaly and edema. These results highlight the system's potential to enhance diagnostic accuracy and support continuous learning in radiology.

Dissonance Between Law Courts and the Science of Visual Perception in Medical Imaging

Lawsuits for spending too little time interpreting each radiological image are a vexatious charge to level against a radiologist in medical malpractice court. In this article, we recount two medicolegal cases where the defendant radiologists were accused of missing a life-threatening diagnosis due to not spending enough time reviewing each image. We consider the literature in vision sciences, visual perception in radiology and interpretive biases to demonstrate that using reading speed as evidence of negligence in a malpractice court represents in incorrect understanding of how radiologists perceive images, including three-dimensional volumetric studies.

Interobserver agreement during clinical magnetic resonance imaging of the equine foot

BACKGROUND

Agreement between experienced observers for assessment of pathology and assessment confidence are poorly documented for magnetic resonance imaging (MRI) of the equine foot.

OBJECTIVES

To report interobserver agreement for pathology assessment and observer confidence for key anatomical structures of the equine foot during MRI.

STUDY DESIGN

Exploratory clinical study.

METHODS

Ten experienced observers (diploma or associate level) assessed 15 equine foot MRI studies acquired from clinical databases of 3 MRI systems. Observers graded pathology in seven key anatomical structures (Grade 1: no pathology, Grade 2: mild pathology, Grade 3: moderate pathology, Grade 4: severe pathology) and provided a grade for their confidence for each pathology assessment (Grade 1: high confidence, Grade 2: moderate confidence, Grade 3: limited confidence, Grade 4: no confidence). Interobserver agreement for the presence/absence of pathology and agreement for individual grades of pathology were assessed with Fleiss' kappa (k). Overall interobserver agreement for pathology was determined using Fleiss' kappa and Kendall's coefficient of concordance (KCC). The distribution of grading was also visualised with bubble charts.

RESULTS

Interobserver agreement for the presence/absence of pathology of individual anatomical structures was poor-to-fair, except for the navicular bone which had moderate agreement (k = 0.52). Relative agreement for pathology grading (accounting for the ranking of grades) ranged from KCC = 0.19 for the distal interphalangeal joint to KCC = 0.70 for the navicular bone. Agreement was generally greatest at the extremes of pathology. Observer confidence in pathology assessment was generally moderate to high.

MAIN LIMITATIONS

Distribution of pathology varied between anatomical structures due to random selection of clinical MRI studies. Observers had most experience with low-field MRI.

CONCLUSIONS

Even with experienced observers, there can be notable variation in the perceived severity of foot pathology on MRI for individual cases, which could be important in a clinical context.

Why did Rubens add a parrot to Titian's The Fall of Man? A pictorial manipulation of joint attention

Almost 400 years ago, Rubens copied Titian's The Fall of Man, albeit with important changes. Rubens altered Titian's original composition in numerous ways, including by changing the gaze directions of the depicted characters and adding a striking red parrot to the painting. Here, we quantify the impact of Rubens's choices on the viewer's gaze behavior. We displayed digital copies of Rubens's and Titian's artworks-as well as a version of Rubens's painting with the parrot digitally removed-on a computer screen while recording the eye movements produced by observers during free visual exploration of each image. To assess the effects of Rubens's changes to Titian's composition, we directly compared multiple gaze parameters across the different images. We found that participants gazed at Eve's face more frequently in Rubens's painting than in Titian's. In addition, gaze positions were more tightly focused for the former than for the latter, consistent with different allocations of viewer interest. We also investigated how gaze fixation on Eve's face affected the perceptual visibility of the parrot in Rubens's composition and how the parrot's presence versus its absence impacted gaze dynamics. Taken together, our results demonstrate that Rubens's critical deviations from Titian's painting have powerful effects on viewers' oculomotor behavior.

Enhancing visual perceptual learning using transcranial electrical stimulation: Transcranial alternating current stimulation outperforms both transcranial direct current and random noise stimulation

Diverse strategies can be employed to enhance visual skills, including visual perceptual learning (VPL) and transcranial electrical stimulation (tES). Combining VPL and tES is a popular method that holds promise for producing significant improvements in visual acuity within a short time frame. However, there is still a lack of comprehensive evaluation regarding the effects of combining different types of tES and VPL on enhancing visual function, especially with a larger sample size. In the present study, we recruited four groups of subjects (26 subjects each) to learn an orientation discrimination task with five daily training sessions. During training, the occipital region of each subject was stimulated by one type of tES-anodal transcranial direct current stimulation (tDCS), alternating current stimulation (tACS) at 10 Hz, high-frequency random noise stimulation (tRNS), and sham tACS-while the subject performed the training task. We found that, compared with the sham stimulation, both the high-frequency tRNS and the 10-Hz tACS facilitated VPL efficiently in terms of learning rate and performance improvement, but there was little modulatory effect in the anodal tDCS condition. Remarkably, the 10-Hz tACS condition exhibited superior modulatory effects compared with the tRNS condition, demonstrating the strongest modulation among the most commonly used tES types for further enhancing vision when combined with VPL. Our results suggest that alpha oscillations play a vital role in VPL. Our study provides a practical guide for vision rehabilitation.

Towards AI-augmented radiology education: a web-based application for perception training in chest X-ray nodule detection

OBJECTIVES

Artificial intelligence (AI)-based applications for augmenting radiological education are underexplored. Prior studies have demonstrated the effectiveness of simulation in radiological perception training. This study aimed to develop and make available a pure web-based application called Perception Trainer for perception training in lung nodule detection in chest X-rays.

METHODS

Based on open-access data, we trained a deep-learning model for lung segmentation in chest X-rays. Subsequently, an algorithm for artificial lung nodule generation was implemented and combined with the segmentation model to allow on-the-fly procedural insertion of lung nodules in chest X-rays. This functionality was integrated into an existing zero-footprint web-based DICOM viewer, and a dynamic HTML page was created to specify case generation parameters.

RESULTS

The result is an easily accessible platform-agnostic web application available at: https://castlemountain.dk/mulrecon/perceptionTrainer.html.The application allows the user to specify the characteristics of lung nodules to be inserted into chest X-rays, and it produces automated feedback regarding nodule detection performance. Generated cases can be shared through a uniform resource locator.

CONCLUSION

We anticipate that the description and availability of our developed solution with open-sourced codes may help facilitate radiological education and stimulate the development of similar AI-augmented educational tools.

ADVANCES IN KNOWLEDGE

A web-based application applying AI-based techniques for radiological perception training was developed. The application demonstrates a novel approach for on-the-fly generation of cases in chest X-ray lung nodule detection employing deep-learning-based segmentation and lung nodule simulation.

Eye tracking: empirical foundations for a minimal reporting guideline

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "An empirically based minimal reporting guideline").

What the Neuroscience and Psychology of Magic Reveal about Misinformation

When we believe misinformation, we have succumbed to an illusion: our perception or interpretation of the world does not match reality. We often trust misinformation for reasons that are unrelated to an objective, critical interpretation of the available data: Key facts go unnoticed or unreported. Overwhelming information prevents the formulation of alternative explanations. Statements become more believable every time they are repeated. Events are reframed or given "spin" to mislead audiences. In magic shows, illusionists apply similar techniques to convince spectators that false and even seemingly impossible events have happened. Yet, many magicians are "honest liars," asking audiences to suspend their disbelief only during the performance, for the sole purpose of entertainment. Magic misdirection has been studied in the lab for over a century. Psychological research has sought to understand magic from a scientific perspective and to apply the tools of magic to the understanding of cognitive and perceptual processes. More recently, neuroscientific investigations have also explored the relationship between magic illusions and their underlying brain mechanisms. We propose that the insights gained from such studies can be applied to understanding the prevalence and success of misinformation. Here, we review some of the common factors in how people experience magic during a performance and are subject to misinformation in their daily lives. Considering these factors will be important in reducing misinformation and encouraging critical thinking in society.

Evaluating artificial intelligence algorithms for use in veterinary radiology

Artificial intelligence is increasingly being used for applications in veterinary radiology, including detection of abnormalities and automated measurements. Unlike human radiology, there is no formal regulation or validation of AI algorithms for veterinary medicine and both general practitioner and specialist veterinarians must rely on their own judgment when deciding whether or not to incorporate AI algorithms to aid their clinical decision-making. The benefits and challenges to developing clinically useful and diagnostically accurate AI algorithms are discussed. Considerations for the development of AI research projects are also addressed. A framework is suggested to help veterinarians, in both research and clinical practice contexts, assess AI algorithms for veterinary radiology.

Deep-learning-based generation of synthetic 6-minute MRI from 2-minute MRI for use in head and neck cancer radiotherapy

Background

Quick magnetic resonance imaging (MRI) scans with low contrast-to-noise ratio are typically acquired for daily MRI-guided radiotherapy setup. However, for patients with head and neck (HN) cancer, these images are often insufficient for discriminating target volumes and organs at risk (OARs). In this study, we investigated a deep learning (DL) approach to generate high-quality synthetic images from low-quality images.

Methods

We used 108 unique HN image sets of paired 2-minute T2-weighted scans (2mMRI) and 6-minute T2-weighted scans (6mMRI). 90 image sets (~20,000 slices) were used to train a 2-dimensional generative adversarial DL model that utilized 2mMRI as input and 6mMRI as output. Eighteen image sets were used to test model performance. Similarity metrics, including the mean squared error (MSE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) were calculated between normalized synthetic 6mMRI and ground-truth 6mMRI for all test cases. In addition, a previously trained OAR DL auto-segmentation model was used to segment the right parotid gland, left parotid gland, and mandible on all test case images. Dice similarity coefficients (DSC) were calculated between 2mMRI and either ground-truth 6mMRI or synthetic 6mMRI for each OAR; two one-sided t-tests were applied between the ground-truth and synthetic 6mMRI to determine equivalence. Finally, a visual Turing test using paired ground-truth and synthetic 6mMRI was performed using three clinician observers; the percentage of images that were correctly identified was compared to random chance using proportion equivalence tests.

Results

The median similarity metrics across the whole images were 0.19, 0.93, and 33.14 for MSE, SSIM, and PSNR, respectively. The median of DSCs comparing ground-truth vs. synthetic 6mMRI auto-segmented OARs were 0.86 vs. 0.85, 0.84 vs. 0.84, and 0.82 vs. 0.85 for the right parotid gland, left parotid gland, and mandible, respectively (equivalence p<0.05 for all OARs). The percent of images correctly identified was equivalent to chance (p<0.05 for all observers).

Conclusions

Using 2mMRI inputs, we demonstrate that DL-generated synthetic 6mMRI outputs have high similarity to ground-truth 6mMRI, but further improvements can be made. Our study facilitates the clinical incorporation of synthetic MRI in MRI-guided radiotherapy.

Negative cues minimize visual search specificity effects

Prior target knowledge (i.e., positive cues) improves visual search performance. However, there is considerable debate about whether distractor knowledge (i.e., negative cues) can guide search. Some studies suggest the active suppression of negatively cued search items, while others suggest the initial capture of attention by negatively cued items. Prior work has used pictorial or specific text cues but has not explicitly compared them. We build on that work by comparing positive and negative cues presented pictorially and as categorical text labels using photorealistic objects and eye movement measures. Search displays contained a target (cued on positive trials), a lure from the target category (cued on negative trials), and four categorically-unrelated distractors. Search performance with positive cues resulted in stronger attentional guidance and faster object recognition for pictorial relative to categorical cues (i.e., a pictorial advantage, suggesting specific visual details afforded by pictorial cues improved search). However, in most search performance metrics, negative cues mitigate the pictorial advantage. Given that the negatively cued items captured attention, generated target guidance but mitigated the pictorial advantage, these results are partly consistent with both existing theories. Specific visual details provided in positive cues produce a large pictorial advantage in all measures, whereas specific visual details in negative cues only produce a small pictorial advantage for object recognition but not for attentional guidance. This asymmetry in the pictorial advantage suggests that the down-weighting of specific negatively cued visual features is less efficient than the up-weighting of specific positively cued visual features.

Mandating Limits on Workload, Duty, and Speed in Radiology

Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.

Enhancement of visual perception by combining transcranial electrical stimulation and visual perceptual training

The visual system remains highly malleable even after its maturity or impairment. Our visual function can be enhanced through many ways, such as transcranial electrical stimulation (tES) and visual perceptual learning (VPL). TES can change visual function rapidly, but its modulation effect is short-lived and unstable. By contrast, VPL can lead to a substantial and long-lasting improvement in visual function, but extensive training is typically required. Theoretically, visual function could be further improved in a shorter time frame by combining tES and VPL than by solely using tES or VPL. Vision enhancement by combining these two methods concurrently is both theoretically and practically significant. In this review, we firstly introduced the basic concept and possible mechanisms of VPL and tES; then we reviewed the current research progress of visual enhancement using the combination of two methods in both general and clinical population; finally, we discussed the limitations and future directions in this field. Our review provides a guide for future research and application of vision enhancement and restoration by combining VPL and tES.

Visual Illusions in Radiology: Untrue Perceptions in Medical Images and Their Implications for Diagnostic Accuracy

Errors in radiologic interpretation are largely the result of failures of perception. This remains true despite the increasing use of computer-aided detection and diagnosis. We surveyed the literature on visual illusions during the viewing of radiologic images. Misperception of anatomical structures is a potential cause of error that can lead to patient harm if disease is seen when none is present. However, visual illusions can also help enhance the ability of radiologists to detect and characterize abnormalities. Indeed, radiologists have learned to exploit certain perceptual biases in diagnostic findings and as training tools. We propose that further detailed study of radiologic illusions would help clarify the mechanisms underlying radiologic performance and provide additional heuristics to improve radiologist training and reduce medical error.