Interpretation of radiological images: towards a framework of knowledge and skills

Twelve tips for applying the think-aloud method to capture cognitive processes

The think-aloud method is an established technique for studying human thought (cognitive) processes. Problem-solving and decision-making are essential skills for medical professionals, and the cognitive processes underlying these skills are complex. Studying these thought processes would enable educators, clinicians, and researchers to modify or refine their approaches and interventions. The think-aloud method has been utilized for capturing cognitive processes in a variety of fields, including computer usability, sports and cognitive psychology. Medical education also recognizes thought processes as valuable data for research and education. This article aims to guide researchers and educators through the preparation and implementation of a think-aloud method to record participants' thought processes during an activity.

Multiparametric MRI for characterization of the tumour microenvironment

Our understanding of tumour biology has evolved over the past decades and cancer is now viewed as a complex ecosystem with interactions between various cellular and non-cellular components within the tumour microenvironment (TME) at multiple scales. However, morphological imaging remains the mainstay of tumour staging and assessment of response to therapy, and the characterization of the TME with non-invasive imaging has not yet entered routine clinical practice. By combining multiple MRI sequences, each providing different but complementary information about the TME, multiparametric MRI (mpMRI) enables non-invasive assessment of molecular and cellular features within the TME, including their spatial and temporal heterogeneity. With an increasing number of advanced MRI techniques bridging the gap between preclinical and clinical applications, mpMRI could ultimately guide the selection of treatment approaches, precisely tailored to each individual patient, tumour and therapeutic modality. In this Review, we describe the evolving role of mpMRI in the non-invasive characterization of the TME, outline its applications for cancer detection, staging and assessment of response to therapy, and discuss considerations and challenges for its use in future medical applications, including personalized integrated diagnostics.

AI in medical diagnosis: AI prediction & human judgment

AI has long been regarded as a panacea for decision-making and many other aspects of knowledge work; as something that will help humans get rid of their shortcomings. We believe that AI can be a useful asset to support decision-makers, but not that it should replace decision-makers. Decision-making uses algorithmic analysis, but it is not solely algorithmic analysis; it also involves other factors, many of which are very human, such as creativity, intuition, emotions, feelings, and value judgments. We have conducted semi-structured open-ended research interviews with 17 dermatologists to understand what they expect from an AI application to deliver to medical diagnosis. We have found four aggregate dimensions along which the thinking of dermatologists can be described: the ways in which our participants chose to interact with AI, responsibility, 'explainability', and the new way of thinking (mindset) needed for working with AI. We believe that our findings will help physicians who might consider using AI in their diagnosis to understand how to use AI beneficially. It will also be useful for AI vendors in improving their understanding of how medics want to use AI in diagnosis. Further research will be needed to examine if our findings have relevance in the wider medical field and beyond.

What's in the Shadows? Formal Analysis: Art History Method to Improve Interpretation Skills for Mammography and Chest Radiographs in Resident Education

RATIONALE AND OBJECTIVES

To assess the feasibility of using an art history tool of formal analysis in resident education for interpretations of mammography and chest radiographs METHODS: In a pre-post study design, residents were shown pre-selected 10 mammograms and 10 chest radiographs for a total of 20 unique anonymized patient cases. After the pretest, residents attended four formal analysis art history lessons. The formal elements included line, light, dark, shade, proportion and balance. The post-tests were administered utilizing the same set of images given during the pre-test. After the completion of the pre- and post-tests, the participants filled out a written survey.

RESULTS

On average, participants improved their image descriptions for a mean of 30% of the total number of mammographic images they evaluated, and similarly they improved their image diagnoses for a mean of 31% of the mammographic images. On average, participants improved their image descriptions for a mean of 37% of the total number of chest radiographs they evaluated, while improving their image diagnoses for a mean of 52% of the chest X-rays. Of the 20 participants, 14 (70%) completed the post study survey. Almost all of the respondents endorsed agreement in understanding the application of formal analysis to radiologic interpretation. Eight out of 14 (57%) participants self-reported improvement in identifying abnormalities and contriving descriptors.

CONCLUSION

This pilot study shows that formal art analysis used by art historians may improve radiologic learners' ability to perceive and describe relevant radiologic abnormalities which in turn would lead to a more accurate radiologic differential diagnosis. The formal analysis process trains the eye to detect and categorize the underlying structure of images. This method provides an alternative arts intervention specifically designed to improve fundamental visual skills in radiology education.

A review and perspective on the neural basis of radiological expertise

Radiological expertise requires tremendous time, effort, and training. While there has been a myriad of studies focusing on radiological expertise and error, the precise underlying neural mechanism still remains largely unexplored. In this article, we review potential neural mechanisms, namely, the fusiform face area, working memory, and predictive coding and propose experiments to test the predictive coding framework.

When Not to Operate on Acute Cases-A Surgeon's Perspective on Rapid Assessment of Emergency Abdominopelvic Computed Tomography

Clinical problem solving evolves in parallel with advances in technology and discoveries in the medical field. However, it always reverts to basic cognitive processes involved in critical thinking, such as hypothetical-deductive reasoning, pattern recognition, and compilation models. When dealing with cases of acute abdominal pain, clinicians should employ all available tools that allow them to rapidly refine their analysis for a definitive diagnosis. Therefore, we propose a standardized method for the quick assessment of abdominopelvic computed tomography as a supplement to the traditional clinical reasoning process. This narrative review explores the cognitive basis of errors in reading imaging. It explains the practical use of attenuation values, contrast phases, and windowing for non-radiologists and details a multistep protocol for finding radiological cues during CT reading and interpretation. This systematic approach describes the salient features and technical tools needed to ascertain the causality between clinical patterns and abdominopelvic changes visible on CT scans from a surgeon's perspective. It comprises 16 sections that should be read successively and that cover the entire abdominopelvic region. Each section details specific radiological signs and provides clear explanations for targeted searches, as well as anatomical and technical hints. Reliance on imaging in clinical problem solving does not make a decision dichotomous nor does it guarantee success in diagnostic endeavors. However, it contributes exact information for supporting the clinical assessments even in the most subtle and intricate conditions.

Impact of case complexity on paediatric dentists' ability to radiographically diagnose traumatic dental injuries

BACKGROUND/AIMS

Being able to correctly interpret radiographs after a traumatic dental injury is an essential skill for providing appropriate and timely treatment. The aim of this study was to assess the impact of case complexity on paediatric dentists' performance when radiographically diagnosing traumatic dental injuries (TDI) and to investigate a possible added value of cone-beam computed tomography (CBCT) when compared with digital intra-oral radiography (2D vs 3D).

MATERIALS AND METHODS

A test panel of paediatric dentists was instructed to detect, identify and interpret radiographic findings using either 2D or 3D images. Intra-oral radiographs and CBCT images of 20 trauma cases were presented in random order, and the findings were recorded using structured scoring sheets. Case complexity was determined by two experienced benchmark scorers. Results were analysed using generalized linear mixed modelling.

RESULTS

In general, performance for detection, identification and interpretation of findings was low, both with 2D and 3D images, with significantly lower values for difficult cases (p < 0.05). For easy as well as for difficult cases, 3D imaging resulted in a significantly better performance for detection and identification of findings (P < 0.001). This was not the case for correct interpretation, where significantly poorer performance was seen when using 3D images for difficult cases (p < 0.05).

CONCLUSION

This study provides evidence that case complexity of traumatic dental injuries influences diagnostic performance. The use of CBCT enhanced detection and identification of findings but when case complexity increased, 3D imaging adversely affected correct interpretation.

Qualitative content analysis of image interpretation education in UK pre-registration diagnostic radiography programmes

INTRODUCTION

Image interpretation is a required capability for all UK pre-registration programmes in diagnostic radiography to meet the needs of graduate practice. It also provides a potential educational foundation for future advanced clinical practice. The aim of this study was to explore how image interpretation education is designed, delivered, and assessed within contemporary UK pre-registration diagnostic radiography programmes.

METHODS

Qualitative content analysis of open-source image interpretation curriculum data extracted from UK Higher Education Institute (HEI) websites.

RESULTS

Extracted search data was initially coded and three overarching themes emerged, image interpretation education vision, operationalisation, and delivery and assessment.

CONCLUSION

This study identified significant heterogeneity in all aspects of UK pre-registration image interpretation education which may suggest an equal heterogeneity can be expected in the image interpretation knowledge, skill, confidence between newly registered practitioners.

IMPLICATIONS FOR PRACTICE

There may be a need for clearer expectations on HEIs by professional and regulatory bodies to ensure consistency in pre-registration image interpretation education.

Impact of compulsory participation of medical students in a multiuser online game to learn radiological anatomy and radiological signs within the virtual world Second Life

Competitive game-based learning within Second Life enables effective teaching of basic radiological anatomy and radiological signs to medical students, with good acceptance and results when students participate voluntarily, but unknown in a compulsory context. The objectives of this study were to reproduce a competitive online game based on self-guided presentations and multiple-choice tests in a mandatory format, to evaluate its development and student perceptions compared to a voluntary edition in 2015 (N = 90). In 2016 and 2017, respectively, 191 and 182 third-year medical students participated in the game as a mandatory course activity. The mean (±SD) score of the game was 74.7% (±19.5%) in 2015, 71.2% (±21.5%) in 2016, and 67.5% (±21.5%) in 2017 (P < 0.01). Participants valued positively the organization and educational contents but found the virtual world less attractive and the game less interesting than in the voluntary edition. The experience globally was rated with 8.2 (±1.5), 7.8 (±1.5), and 7.1 (±1.7) mean points (±SD) in a ten-point scale, in the 2015, 2016, and 2017 editions, respectively (P < 0.05). Competitive learning games within virtual worlds like Second Life have great learning potential in radiology, but the mean score in the game decreased, acceptance of virtual world technology was lower, and opinion about the game was worse with a compulsory participation, and even worse when dropouts were not allowed. Under the conditions in which this study was conducted, learning games in three-dimensional virtual environments should be voluntary to maintain adequate motivation and engagement of medical students.

Improving Radiology Trainees' Perception Using Where's Waldo?

RATIONALE AND OBJECTIVES

Perception is an essential skill leading to expertise in diagnostic radiology. We determined if practicing "Where's Waldo?" images improves accuracy and speed with which first and second year radiology residents detect abnormalities on chest radiographs (CXRs).

MATERIALS AND METHODS

Residents at three institutions were pretested using 50 CXRs, identifying locations of potential abnormalities. They were then split into trained (examining 7 "Where's Waldo?" images over three weeks) and control groups (no "Where's Waldo?"). They were then re-tested on the 50 CXRs. At one site, visual search parameters were acquired. Data were analyzed with repeated measures ANOVAs.

RESULTS

There was no significant difference in performance for trained vs control (F = 0.622, p = 0.436), with both improving significantly on post-test (F = 4.72, p = 0.037). Session time decreased significantly for both groups from pre to post-test (F = 81.47, p < 0.0001) and the decrease was significantly more (F = 31.59, p < 0.0001) for the trained group than the control group as well as for PGY with PGY3 having a larger average decrease in session time than PGY2. Eye-tracking data also showed significant increases in per image search efficiency with training.

CONCLUSION

Practicing "Where's Waldo?" or similar nonradiology search tasks may facilitate the acquisition of radiology image search but not detection skills, impacting reading efficiency more than detection accuracy.

Lessons from the video-EEG telemetry unit

Epilepsy is a clinical diagnosis, based primarily on patient and witness histories. Where there is diagnostic uncertainty or when epilepsy surgery is being considered, long-term video-EEG monitoring in a telemetry unit remains the gold standard investigation for diagnostic clarification or presurgical localisation. We present six illustrative cases, highlighting important points that emerged during video-EEG review including potential pitfalls in video-EEG interpretation, and how the investigation helped with diagnosis and subsequent management. The diagnostic process strongly emphasises seizure semiology, more so than EEG.

Characteristics of expert search behavior in volumetric medical image interpretation

Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.

Expertise development in volumetric image interpretation of radiology residents: what do longitudinal scroll data reveal?

The current study used theories on expertise development (the holistic model of image perception and the information reduction hypothesis) as a starting point to identify and explore potentially relevant process measures to monitor and evaluate expertise development in radiology residency training. It is the first to examine expertise development in volumetric image interpretation (i.e., CT scans) within radiology residents using scroll data collected longitudinally over five years of residency training. Consistent with the holistic model of image perception, the percentage of time spent on full runs, i.e. scrolling through more than 50% of the CT-scan slices (global search), decreased within residents over residency training years. Furthermore, the percentage of time spent on question-relevant areas in the CT scans increased within residents over residency training years, consistent with the information reduction hypothesis. Second, we examined if scroll patterns can predict diagnostic accuracy. The percentage of time spent on full runs and the percentage of time spent on question-relevant areas did not predict diagnostic accuracy. Thus, although scroll patterns over training years are consistent with visual expertise theories, they could not be used as predictors of diagnostic accuracy in the current study. Therefore, the relation between scroll patterns and performance needs to be further examined, before process measures can be used to monitor and evaluate expertise development in radiology residency training.

The Search Patterns of Abdominal Imaging Subspecialists for Abdominal Computed Tomography: Toward a Foundational Pattern for New Radiology Residents

Objectives:

The routine search patterns used by subspecialty abdominal imaging experts to inspect the image volumes of abdominal/pelvic computed tomography (CT) have not been well characterized or rendered in practical or teachable terms. The goal of this study is to describe the search patterns used by experienced subspecialty imagers when reading a normal abdominal CT at a modern picture archiving and communication system workstation, and utilize this information to propose guidelines for residents as they learn to interpret CT during training.

Material and Methods:

Twenty-two academic subspecialists enacted their routine search pattern on a normal contrast-enhanced abdominal/pelvic CT study under standardized display parameters. Readers were told that the scan was normal and then asked to verbalize where their gaze centered and moved through the axial, coronal, and sagittal image stacks, demonstrating eye position with a cursor as needed. A peer coded the reported eye gaze movements and scrilling behavior. Spearman correlation coefficients were calculated between years of professional experience and the numbers of passes through the lung bases, liver, kidneys, and bowel.

Results:

All readers followed an initial organ-by-organ approach. Larger organs were examined by drilling, while smaller organs by oscillation or scanning. Search elements were classified as drilling, scanning, oscillation, and scrilling (scan drilling); these categories were parsed as necessary. The greatest variability was found in the examination the body wall and bowel/mesentery. Two modes of scrilling were described, and these classified as roaming and zigzagging. The years of experience of the readers did not correlated to number of passes made through the lung bases, liver, kidneys, or bowel.

Conclusion:

Subspecialty abdominal radiologists negotiate through the image stacks of an abdominal CT study in broadly similar ways. Collation of the approaches suggests a foundational search pattern for new trainees.

Impact of a web-based module on trainees' ability to interpret neonatal cranial ultrasound

BACKGROUND

Accurate interpretations of neonatal cranial ultrasound (CUS) studies are essential skills for physicians in neonatal intensive care units (NICUs) in order to properly diagnose and manage brain injury. However, these skills are not formally taught to pediatric and neonatal-perinatal medicine (NPM) trainees in Canada. Therefore, our study describes the design, implementation, and evaluation of a new web-based learning (WBL) module that focuses on teaching these skills.

METHODS

Trainees' needs assessment survey, sent to all NPM and pediatrics trainees (n = 62), concluded that most of them feel uncomfortable with their ability to interpret CUS, highlighting the need for a new educational intervention. The needs assessment informed the development of the WBL module, which we evaluated using questionnaires and pre-and post-testing methods to measure participants' satisfaction, knowledge gain, skills development, and behaviour changes. Only trainees rotating through the NICU over 6 months (n = 23) were invited to participate in all the evaluation steps. We used the ADDIE instructional design model as a framework for this project.

RESULTS

Respondents were very satisfied with the module, and their baseline knowledge increased significantly after studying and engaging with the module. The post-test score was 76% (p < 0.001) compared to the pre-test mean score of 42%. Tests for CUS interpretation skills assessment showed that 49% of pre-test answers were incorrect compared to 8% in the post-test (p < 0.001). Seventy-eight percent of trainees (n = 18) responded to a survey conducted a year after implementation, and 78% of the respondents (n = 14) reported that they still used these skills and shared this knowledge with junior trainees.

CONCLUSION

A WBL module for teaching neonatal CUS interpretation considerably improved trainees' knowledge and enhanced their skills in interpreting neonatal CUS.

Perceptual learning in the identification of lung cancer in chest radiographs

Extensive research has shown that practice yields highly specific perceptual learning of simple visual properties such as orientation and contrast. Does this same learning characterize more complex perceptual skills? Here we investigated perceptual learning of complex medical images. Novices underwent training over four sessions to discriminate which of two chest radiographs contained a tumor and to indicate the location of the tumor. In training, one group received six repetitions of 30 normal/abnormal images, the other three repetitions of 60 normal/abnormal images. Groups were then tested on trained and novel images. To assess the nature of perceptual learning, test items were presented in three formats – the full image, the cutout of the tumor, or the background only. Performance improved across training sessions, and notably, the improvement transferred to the classification of novel images. Training with more repetitions on fewer images yielded comparable transfer to training with fewer repetitions on more images. Little transfer to novel images occurred when tested with just the cutout of the cancer region or just the background, but a larger cutout that included both the cancer region and some surrounding regions yielded good transfer. Perceptual learning contributes to the acquisition of expertise in cancer image perception.

A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence

As the first step in image interpretation is detection, an error in perception can prematurely end the diagnostic process leading to missed diagnoses. Because perceptual errors of this sort-"failure to detect"-are the most common interpretive error (and cause of litigation) in radiology, understanding the nature of perceptual expertise is essential in decreasing radiology's long-standing error rates. In this article, we review what constitutes a perceptual error, the existing models of radiologic image perception, the development of perceptual expertise and how it can be tested, perceptual learning methods in training radiologists, and why understanding perceptual expertise is still relevant in the era of artificial intelligence. Adding targeted interventions, such as perceptual learning, to existing teaching practices, has the potential to enhance expertise and reduce medical error.

Introduction of Ultrasound Simulation in Medical Education: Exploratory Study

BACKGROUND

Ultrasound is ubiquitous across all disciplines of medicine; it is one of the most commonly used noninvasive, painless diagnostic tools. However, not many are educated and trained well enough in its use. Ultrasound requires not only theoretical knowledge but also extensive practical experience. The simulated setting offers the safest environment for health care professionals to learn and practice using ultrasound.

OBJECTIVE

This study aimed to (1) assess health care professionals' need for and enthusiasm toward practicing using ultrasound via simulation and (2) gauge their perception and acceptance of simulation as an integral element of ultrasound education in medical curricula.

METHODS

A day-long intervention was organized at the American University of Beirut Medical Center (AUBMC) to provide a free-of-charge interactive ultrasound simulation workshop-using CAE Vimedix high-fidelity simulator-for health care providers, including physicians, nurses, ultrasound technicians, residents, and medical students. Following the intervention, attendees completed an evaluation, which included 4 demographic questions and 16 close-ended questions based on a Likert scale agree-neutral-disagree. The results presented are based on this evaluation form.

RESULTS

A total of 41 participants attended the workshop (46% [19/41] physicians, 30% [12/41] residents, 19% [8/41] sonographers, and 5% [2/41] medical students), mostly from AUBMC (88%, 36/41), with an average experience of 2.27 (SD 3.45) years and 30 (SD 46) scans per attendee. Moreover, 15 out of 41 (36%) participants were from obstetrics and gynecology, 11 (27%) from internal medicine, 4 (10%) from pediatrics, 4 (10%) from emergency medicine, 2 (5%) from surgery and family medicine, and 5 (12%) were technicians. The majority of participants agreed that ultrasound provided a realistic setting (98%, 40/41) and that it allowed for training and identification of pathologies (88%, 36/41). Furthermore, 100% (41/41) of the participants agreed that it should be part of the curriculum either in medical school or residency, and most of the participants approved it for training (98%, 40/41) and teaching (98%, 40/41).

CONCLUSIONS

All attendees were satisfied with the intervention. There was a positive perception toward the use of simulation for training and teaching medical students and residents in using ultrasound, and there was a definite need and enthusiasm for its integration into curricula. Simulation offers an avenue not only for teaching but also for practicing the ultrasound technology by both medical students and health care providers.

Analysis of Perceptual Expertise in Radiology - Current Knowledge and a New Perspective

Radiologists rely principally on visual inspection to detect, describe, and classify findings in medical images. As most interpretive errors in radiology are perceptual in nature, understanding the path to radiologic expertise during image analysis is essential to educate future generations of radiologists. We review the perceptual tasks and challenges in radiologic diagnosis, discuss models of radiologic image perception, consider the application of perceptual learning methods in medical training, and suggest a new approach to understanding perceptional expertise. Specific principled enhancements to educational practices in radiology promise to deepen perceptual expertise among radiologists with the goal of improving training and reducing medical error.

Development of competence in volumetric image interpretation in radiology residents

BACKGROUND

During residency, radiology residents learn to interpret volumetric radiological images. The development of their competence for volumetric image interpretation, as opposed to 2D image reading, is not completely understood. The purpose of the present study was to investigate how competence for volumetric image interpretation develops in radiology residents and how this compares with competence development for 2D image interpretation, by studying resident scores on image-based items in digital radiology tests.

METHODS

We reviewed resident scores on volumetric and 2D image-based test items in 9 consecutive semi-annual digital radiology tests that were carried out from November 2013 to April 2018. We assessed percentage-correct sum scores for all test items about volumetric images and for all test items about 2D images in each test as well as for all residents across the 9 tests (i.e. 4.5 years of test materials). We used a paired t-test to analyze whether scores differed between volumetric and 2D image-based test items in individual residents in postgraduate year (PGY) 0-5, subdivided in 10 half-year phases (PGY 0-0.5, 0.5-1.0, 1.0-1.5 et cetera).

RESULTS

The percentage-correct scores on volumetric and 2D image-based items showed a comparable trend of development, increasing in the first half of residency and flattening off in the second half. Chance-corrected scores were generally lower in volumetric than in 2D items (on average 1-5% points). In PGY 1.5-4.5, this score difference was statistically significant (p-values ranging from 0.02 to < 0.001), with the largest difference found in PGY 2.5 (mean: 5% points; 95% CI: -7.3 - -3.4). At the end of training in PGY 5, there was no statistically significant score difference between both item types.

CONCLUSIONS

The development of competence in volumetric image interpretation fits a similar curvilinear growth curve during radiology residency as 2D image interpretation competence in digital radiology tests. Although residents performed significantly lower on volumetric than 2D items in PGY 1.5-4.5, we consider the magnitude of this difference as relatively small for our educational setting and we suggest that throughout radiology training there are no relevant differences in the development of both types of competences, as investigated by digital radiology tests.

The Role of Anatomy Computer-Assisted Learning on Spatial Abilities of Medical Students

Currently, medical education context poses different challenges to anatomy, contributing to the introduction of new pedagogical approaches, such as computer-assisted learning (CAL). This approach provides insight into students' learning profiles and skills that enhance anatomy knowledge acquisition. To understand the influence of anatomy CAL on spatial abilities, a study was conducted. A total of 671 medical students attending Musculoskeletal (MA) and Cardiovascular Anatomy (CA) courses, were allocated to one of three groups (MA Group, CA Group, MA + CA Group). Students' pre-training and post-training spatial abilities were assessed through Mental Rotations Test (MRT), with scores ranging between 0-24. After CAL training sessions, students' spatial abilities performance improved (9.72 ± 4.79 vs. 17.05 ± 4.57, P < 0.001). Although male students in both MA Group and CA Group show better baseline spatial abilities, no sex differences were found after CAL training. The improvement in spatial abilities score between sessions (Delta MRT) was correlated with Musculoskeletal Anatomy training sessions in MA Group (r = 0.333, P < 0.001) and MA + CA Group (r = 0.342, P < 0.001), and with Cardiovascular Anatomy training sessions in CA Group (r = 0.461, P = 0.001) and MA + CA Group (r = 0.324, P = 0.001). Multiple linear regression models were used, considering the Delta MRT as dependent variable. An association of Delta MRT to the amount of CAL training and the baseline spatial abilities was observed. The results suggest that CAL training in anatomy has positive dose-dependent effect on spatial abilities.

Analysis of the French national evaluation of radiology residents

PURPOSE

In France, a national evaluation is given annually to radiology residents. The aim of this study was to perform both a docimological analysis of the quality of the questionnaire and a statistical analysis of the results.

MATERIALS AND METHODS

This retrospective study, which included French radiology residents from Year 1 to Year 5 of residency, was performed from 2015 to 2017 across 25 medical universities in France. Both qualitative and quantitative docimological analyses were performed as assessed by the Cronbach alpha coefficient, the difficulty of question (PDI), and the coefficient of discrimination (Rir). Results to the questionnaire were compared between years of residency.

RESULTS

The results of the analysis confirmed the quality of the questionnaire (Cronbach alpha coefficient=0.71, mean [PDI=0.40]) though the majority of questions could be answered by memory rather than cognitive ability. The mean Rir was 0.02, indicating that students could not be certified using only the questionnaire. The results measuring resident level of knowledge were moderate, with mean results ranging from 9.2/20 at the first year to 11.3/20 at the fifth year of residency (P<0.001). There were no significant differences in results obtained between the third, fourth, and fifth year of residency but results were significantly different among university hospitals.

CONCLUSION

Even if close interactions exist between learning and pedagogic environment, our results suggest that it may be useful to further develop an evaluation process in relation with pedagogic instructions in order to provide more optimal training.

Volumetric image interpretation in radiology: scroll behavior and cognitive processes

The interpretation of medical images is a primary task for radiologists. Besides two-dimensional (2D) images, current imaging technologies allow for volumetric display of medical images. Whereas current radiology practice increasingly uses volumetric images, the majority of studies on medical image interpretation is conducted on 2D images. The current study aimed to gain deeper insight into the volumetric image interpretation process by examining this process in twenty radiology trainees who all completed four volumetric image cases. Two types of data were obtained concerning scroll behaviors and think-aloud data. Types of scroll behavior concerned oscillations, half runs, full runs, image manipulations, and interruptions. Think-aloud data were coded by a framework of knowledge and skills in radiology including three cognitive processes: perception, analysis, and synthesis. Relating scroll behavior to cognitive processes showed that oscillations and half runs coincided more often with analysis and synthesis than full runs, whereas full runs coincided more often with perception than oscillations and half runs. Interruptions were characterized by synthesis and image manipulations by perception. In addition, we investigated relations between cognitive processes and found an overall bottom-up way of reasoning with dynamic interactions between cognitive processes, especially between perception and analysis. In sum, our results highlight the dynamic interactions between these processes and the grounding of cognitive processes in scroll behavior. It suggests, that the types of scroll behavior are relevant to describe how radiologists interact with and manipulate volumetric images.

Getting the picture: visual interpretation in ophthalmology residency training

OBJECTIVES

Visual interpretation is essential in many fields of health care. Although diagnostic competency can be measured as an educational outcome, few accounts have addressed tacit aspects of visual interpretation in clinical training; these include the disciplining of the trainee's attentions and the trainee's acculturation into expected styles of communicating visual interpretations to others. This paper describes values and dispositions that are taught to ophthalmology trainees as they learn to reason through visual information, and explores how these qualities are evaluated during residency training.

METHODS

The project was based on 6 months of ethnographic participant observation and interviews in an ophthalmology residency programme. Observational notes and interview transcripts pertaining to visual interpretation were isolated for qualitative analysis in the tradition of sociocultural anthropology, guided by literature on communication in medical education and the socialisation of health professionals.

RESULTS

Residents and faculty members identified visual interpretation as one of the most challenging skills expected of ophthalmology trainees. They expressed a belief that 'systematic' approaches, where visual information is parsed in a stepwise fashion, reduce the chance of trainees overlooking or misinterpreting key diagnostic features. This sensory discipline was represented in narrative form when faculty members asked residents to interpret images aloud, as residents were expected to follow prescribed sequences for describing the content of images before commenting on possible diagnoses.

CONCLUSIONS

Sensory processing is ordinarily opaque to outside observers, but the ritual of describing images in highly regimented narratives allows residents to demonstrate how they gather and reason through visual information. The form of these narratives reflects values that residents are expected to embody during their training, such as being thorough and methodical; it may also serve a pedagogical function by entrenching those values. Further research is needed to characterise how the performance of speech genres shapes the interpretive skills of medical trainees.

Determinants of Difficulty and Discriminating Power of Image-based Test Items in Postgraduate Radiological Examinations

RATIONALE AND OBJECTIVES

The psychometric characteristics of image-based test items in radiological written examinations are not well known. In this study, we explored difficulty and discriminating power of these test items in postgraduate radiological digital examinations.

MATERIALS AND METHODS

We reviewed test items of seven Dutch Radiology Progress Tests (DRPTs) that were taken from October 2013 to April 2017. The DRPT is a semiannual formative examination, required for all Dutch radiology residents. We assessed several stimulus and response characteristics of test items. The response format of test items included true or false, single right multiple choice with 2, 3, 4, or ≥5 answer options, pick-N multiple-choice, drag-and-drop, and long-list-menu formats. We calculated item P values and item-rest-correlation (Rir) values to assess difficulty and discriminating power. We performed linear regression analysis in image-based test items to investigate whether P and Rir values were significantly related to stimulus and response characteristics. Also, we compared psychometric indices between image-based test items and text-alone items.

RESULTS

P and Rir values of image-based items (n = 369) were significantly related to the type of response format (P < .001), and not to which of the seven DRPTs the item was obtained from, radiological subspecialty domain, nonvolumetric or volumetric character of images, or context-rich or context-free character of the stimulus. When accounted for type of response format, difficulty and discriminating power of image-based items did not differ significantly from text-alone items (n = 881). Test items with a relatively large number of answer options were generally more difficult, and discriminated better among high- and low-performing candidates.

CONCLUSION

In postgraduate radiological written examinations, difficulty and discriminating power of image-based test items are related to the type of response format and are comparable to those of text-alone items. We recommend a response format with a relatively large number of answer options to optimize psychometric indices of radiological image-based test items.

"Yes, and …" Exploring the Future of Learning Analytics in Medical Education

This Conversations Starter article presents a selected research abstract from the 2017 Association of American Medical Colleges Northeastern Region Group on Educational Affairs annual spring meeting. The abstract is paired with the integrative commentary of three experts who shared their thoughts stimulated by the study. Commentators brainstormed "what's next" with learning analytics in medical education, including advancements in interaction metrics and the use of interactivity analysis to deepen understanding of perceptual, cognitive, and social learning and transfer processes.

The Holistic Processing Account of Visual Expertise in Medical Image Perception: A Review

In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.

Radiology education: a radiology curriculum for all medical students?

Diagnostic errors in radiology are frequent and can cause severe patient harm. Despite large performance differences between radiologists and non-radiology physicians, the latter often interpret medical images because electronic health records make images available throughout the hospital. Some people argue that non-radiologists should not diagnose medical images at all, and that medical school should focus on teaching ordering skills instead of image interpretation skills. We agree that teaching ordering skills is crucial as most physicians will need to order medical images in their professional life. However, we argue that the availability of medical images is so ubiquitous that it is important that non-radiologists are also trained in the basics of medical image interpretation and, additionally in recognizing when radiological consultancy should be sought. In acute situations, basic image interpretations skills can be life-saving. We plead for a radiology curriculum for all medical students. This should include the interpretation of common abnormalities on chest and skeletal radiographs and a basic distinction of normal from abnormal images. Furthermore, substantial attention should be given to the correct ordering of radiological images. Finally, it is critical that students are trained in deciding when to consult a radiologist.

How visual search relates to visual diagnostic performance: a narrative systematic review of eye-tracking research in radiology

Eye tracking research has been conducted for decades to gain understanding of visual diagnosis such as in radiology. For educational purposes, it is important to identify visual search patterns that are related to high perceptual performance and to identify effective teaching strategies. This review of eye-tracking literature in the radiology domain aims to identify visual search patterns associated with high perceptual performance. Databases PubMed, EMBASE, ERIC, PsycINFO, Scopus and Web of Science were searched using 'visual perception' OR 'eye tracking' AND 'radiology' and synonyms. Two authors independently screened search results and included eye tracking studies concerning visual skills in radiology published between January 1, 1994 and July 31, 2015. Two authors independently assessed study quality with the Medical Education Research Study Quality Instrument, and extracted study data with respect to design, participant and task characteristics, and variables. A thematic analysis was conducted to extract and arrange study results, and a textual narrative synthesis was applied for data integration and interpretation. The search resulted in 22 relevant full-text articles. Thematic analysis resulted in six themes that informed the relation between visual search and level of expertise: (1) time on task, (2) eye movement characteristics of experts, (3) differences in visual attention, (4) visual search patterns, (5) search patterns in cross sectional stack imaging, and (6) teaching visual search strategies. Expert search was found to be characterized by a global-focal search pattern, which represents an initial global impression, followed by a detailed, focal search-to-find mode. Specific task-related search patterns, like drilling through CT scans and systematic search in chest X-rays, were found to be related to high expert levels. One study investigated teaching of visual search strategies, and did not find a significant effect on perceptual performance. Eye tracking literature in radiology indicates several search patterns are related to high levels of expertise, but teaching novices to search as an expert may not be effective. Experimental research is needed to find out which search strategies can improve image perception in learners.

Identifying error types in visual diagnostic skill assessment

BACKGROUND

Misinterpretation of medical images is an important source of diagnostic error. Errors can occur in different phases of the diagnostic process. Insight in the error types made by learners is crucial for training and giving effective feedback. Most diagnostic skill tests however penalize diagnostic mistakes without an eye for the diagnostic process and the type of error. A radiology test with stepwise reasoning questions was used to distinguish error types in the visual diagnostic process. We evaluated the additional value of a stepwise question-format, in comparison with only diagnostic questions in radiology tests.

METHODS

Medical students in a radiology elective (n=109) took a radiology test including 11-13 cases in stepwise question-format: marking an abnormality, describing the abnormality and giving a diagnosis. Errors were coded by two independent researchers as perception, analysis, diagnosis, or undefined. Erroneous cases were further evaluated for the presence of latent errors or partial knowledge. Inter-rater reliabilities and percentages of cases with latent errors and partial knowledge were calculated.

RESULTS

The stepwise question-format procedure applied to 1351 cases completed by 109 medical students revealed 828 errors. Mean inter-rater reliability of error type coding was Cohen's κ=0.79. Six hundred and fifty errors (79%) could be coded as perception, analysis or diagnosis errors. The stepwise question-format revealed latent errors in 9% and partial knowledge in 18% of cases.

CONCLUSIONS

A stepwise question-format can reliably distinguish error types in the visual diagnostic process, and reveals latent errors and partial knowledge.

How Radiologists Think: Understanding Fast and Slow Thought Processing and How It Can Improve Our Teaching

Scholars have identified two distinct ways of thinking. This "Dual Process Theory" distinguishes a fast, nonanalytical way of thinking, called "System 1," and a slow, analytical way of thinking, referred to as "System 2." In radiology, we use both methods when interpreting and reporting images, and both should ideally be emphasized when educating our trainees. This review provides practical tips for improving radiology education, by enhancing System 1 and System 2 thinking among our trainees.

What We Do and Do Not Know about Teaching Medical Image Interpretation

Educators in medical image interpretation have difficulty finding scientific evidence as to how they should design their instruction. We review and comment on 81 papers that investigated instructional design in medical image interpretation. We distinguish between studies that evaluated complete offline courses and curricula, studies that evaluated e-learning modules, and studies that evaluated specific educational interventions. Twenty-three percent of all studies evaluated the implementation of complete courses or curricula, and 44% of the studies evaluated the implementation of e-learning modules. We argue that these studies have encouraging results but provide little information for educators: too many differences exist between conditions to unambiguously attribute the learning effects to specific instructional techniques. Moreover, concepts are not uniformly defined and methodological weaknesses further limit the usefulness of evidence provided by these studies. Thirty-two percent of the studies evaluated a specific interventional technique. We discuss three theoretical frameworks that informed these studies: diagnostic reasoning, cognitive schemas and study strategies. Research on diagnostic reasoning suggests teaching students to start with non-analytic reasoning and subsequently applying analytic reasoning, but little is known on how to train non-analytic reasoning. Research on cognitive schemas investigated activities that help the development of appropriate cognitive schemas. Finally, research on study strategies supports the effectiveness of practice testing, but more study strategies could be applicable to learning medical image interpretation. Our commentary highlights the value of evaluating specific instructional techniques, but further evidence is required to optimally inform educators in medical image interpretation.

Systematic viewing in radiology: seeing more, missing less?

To prevent radiologists from overlooking lesions, radiology textbooks recommend "systematic viewing," a technique whereby anatomical areas are inspected in a fixed order. This would ensure complete inspection (full coverage) of the image and, in turn, improve diagnostic performance. To test this assumption, two experiments were performed. Both experiments investigated the relationship between systematic viewing, coverage, and diagnostic performance. Additionally, the first investigated whether systematic viewing increases with expertise; the second investigated whether novices benefit from full-coverage or systematic viewing training. In Experiment 1, 11 students, ten residents, and nine radiologists inspected five chest radiographs. Experiment 2 had 75 students undergo a training in either systematic, full-coverage (without being systematic) or non-systematic viewing. Eye movements and diagnostic performance were measured throughout both experiments. In Experiment 1, no significant correlations were found between systematic viewing and coverage, r = -.10, p = .62, and coverage and performance, r = -.06, p = .74. Experts were significantly more systematic than students F2,25 = 4.35, p = .02. In Experiment 2, significant correlations were found between systematic viewing and coverage, r = -.35, p < .01, but not between coverage and performance, r = .13, p = .31. Participants in the full-coverage training performed worse compared with both other groups, which did not differ between them, F2,71 = 3.95, p = .02. In conclusion, the data question the assumption that systematic viewing leads to increased coverage, and, consequently, to improved performance. Experts inspected cases more systematically, but students did not benefit from systematic viewing training.

A Vertically Integrated Online Radiology Curriculum Developed as a Cognitive Apprenticeship: Impact on Student Performance and Learning

RATIONALE AND OBJECTIVES

The principles of Collins' cognitive apprenticeship model were used to design a radiology curriculum in which medical students practice radiological skills using online case-based modules. The modules are embedded within clinical third-year clerkships, and students are provided with personalized feedback from the instructors. We describe the development of the vertical online radiology curriculum and evaluate its impact on student achievement and learning process using a mixed method approach.

MATERIALS AND METHODS

The curriculum was developed over a 2-year period. Student participation was voluntary in the first year and mandatory in the second year. For quantitative curriculum evaluation, student metrics for voluntary versus mandatory groups were assessed using independent sample t tests and variable entry method regression analysis. For qualitative analysis, responses from a survey of students about the value of the curriculum were organized into defined themes using consensus coding.

RESULTS

Mandatory participation significantly improved (p = .001) the mean radiology examination score (82 %) compared to the voluntary group (73%), suggesting that mandatory participation had a beneficial effect on student performance. Potential preexisting differences in underlying general academic performance were accounted for by including mean basic science grades as the first variable in the regression model. The significant increase in R(2) from .16 to .28 when number of radiology cases completed was added to the original model, and the greater value of the standardized beta for this variable, suggest that the curriculum made a significant contribution to students' radiology examination scores beyond their baseline academic performance. Five dominant themes about curricular characteristics that enhanced student learning and beneficial outcomes emerged from consensus coding. These themes were (1) self-paced design, (2) receiving feedback from faculty, (3) clinical relevance of cases, (4) gaining confidence in interpreting radiological images, and (5) transfer of conceptual knowledge to actual practice.

CONCLUSIONS

The vertically integrated online radiology curriculum can positively impact student performance and learning process in the context of the cognitive apprenticeship model.

Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial

OBJECTIVE

To study the effect of initial simulation-based transvaginal sonography (TVS) training compared with clinical training only, on the clinical performance of residents in obstetrics and gynecology (Ob-Gyn), assessed 2 months into their residency.

METHODS

In a randomized study, new Ob-Gyn residents (n = 33) with no prior ultrasound experience were recruited from three teaching hospitals. Participants were allocated to either simulation-based training followed by clinical training (intervention group; n = 18) or clinical training only (control group; n = 15). The simulation-based training was performed using a virtual-reality TVS simulator until an expert performance level was attained, and was followed by training on a pelvic mannequin. After 2 months of clinical training, one TVS examination was recorded for assessment of each resident's clinical performance (n = 26). Two ultrasound experts blinded to group allocation rated the scans using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale.

RESULTS

During the 2 months of clinical training, participants in the intervention and control groups completed an average ± SD of 58 ± 41 and 63 ± 47 scans, respectively (P = 0.67). In the subsequent clinical performance test, the intervention group achieved higher OSAUS scores than did the control group (mean score, 59.1% vs 37.6%, respectively; P < 0.001). A greater proportion of the intervention group passed a pre-established pass/fail level than did controls (85.7% vs 8.3%, respectively; P < 0.001).

CONCLUSION

Simulation-based ultrasound training leads to substantial improvement in clinical performance that is sustained after 2 months of clinical training. © 2015 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

Transfer from point-of-care Ultrasonography training to diagnostic performance on patients--a randomized controlled trial

BACKGROUND

Clinicians are increasingly using point-of-care ultrasonography for bedside examinations of patients. However, proper training is needed in this technique, and it is unknown whether the skills learned from focused Ultrasonography courses are being transferred to diagnostic performance on patients.

METHODS

Thirty-one physicians were randomized to participate in a focused Ultrasonography course or control circumstances before they examined 4 patients with different abdominal conditions by ultrasonography. Performance scores and diagnostic accuracy were compared using independent samples t test and binary logistic regression, respectively.

RESULTS

There was a significant difference in the performance score between the intervention group (27.4%) and the control group (18.0%, P = .004) and the diagnostic accuracy between the intervention group (65%) and the control group (39%, P = .014).

CONCLUSIONS

Clinicians could successfully transfer learning from an Ultrasonography course to improve diagnostic performance on patients. However, our results also indicate a need for more training when new technologies such as point-of-care ultrasonography are introduced.

Volumetric and two-dimensional image interpretation show different cognitive processes in learners

RATIONALE AND OBJECTIVES

In current practice, radiologists interpret digital images, including a substantial amount of volumetric images. We hypothesized that interpretation of a stack of a volumetric data set demands different skills than interpretation of two-dimensional (2D) cross-sectional images. This study aimed to investigate and compare knowledge and skills used for interpretation of volumetric versus 2D images.

MATERIALS AND METHODS

Twenty radiology clerks were asked to think out loud while reading four or five volumetric computed tomography (CT) images in stack mode and four or five 2D CT images. Cases were presented in a digital testing program allowing stack viewing of volumetric data sets and changing views and window settings. Thoughts verbalized by the participants were registered and coded by a framework of knowledge and skills concerning three components: perception, analysis, and synthesis. The components were subdivided into 16 discrete knowledge and skill elements. A within-subject analysis was performed to compare cognitive processes during volumetric image readings versus 2D cross-sectional image readings.

RESULTS

Most utterances contained knowledge and skills concerning perception (46%). A smaller part involved synthesis (31%) and analysis (23%). More utterances regarded perception in volumetric image interpretation than in 2D image interpretation (Median 48% vs 35%; z = -3.9; P < .001). Synthesis was less prominent in volumetric than in 2D image interpretation (Median 28% vs 42%; z = -3.9; P < .001). No differences were found in analysis utterances.

CONCLUSIONS

Cognitive processes in volumetric and 2D cross-sectional image interpretation differ substantially. Volumetric image interpretation draws predominantly on perceptual processes, whereas 2D image interpretation is mainly characterized by synthesis. The results encourage the use of volumetric images for teaching and testing perceptual skills.

Support for external validity of radiological anatomy tests using volumetric images

RATIONALE AND OBJECTIVES

Radiology practice has become increasingly based on volumetric images (VIs), but tests in medical education still mainly involve two-dimensional (2D) images. We created a novel, digital, VI test and hypothesized that scores on this test would better reflect radiological anatomy skills than scores on a traditional 2D image test. To evaluate external validity we correlated VI and 2D image test scores with anatomy cadaver-based test scores.

MATERIALS AND METHODS

In 2012, 246 medical students completed one of two comparable versions (A and B) of a digital radiology test, each containing 20 2D image and 20 VI questions. Thirty-three of these participants also took a human cadaver anatomy test. Mean scores and reliabilities of the 2D image and VI subtests were compared and correlated with human cadaver anatomy test scores. Participants received a questionnaire about perceived representativeness and difficulty of the radiology test.

RESULTS

Human cadaver test scores were not correlated with 2D image scores, but significantly correlated with VI scores (r = 0.44, P < .05). Cronbach's α reliability was 0.49 (A) and 0.65 (B) for the 2D image subtests and 0.65 (A) and 0.71 (B) for VI subtests. Mean VI scores (74.4%, standard deviation 2.9) were significantly lower than 2D image scores (83.8%, standard deviation 2.4) in version A (P < .001). VI questions were considered more representative of clinical practice and education than 2D image questions and less difficult (both P < .001).

CONCLUSIONS

VI tests show higher reliability, a significant correlation with human cadaver test scores, and are considered more representative for clinical practice than tests with 2D images.

Volumetric CT-images improve testing of radiological image interpretation skills

RATIONALE AND OBJECTIVES

Current radiology practice increasingly involves interpretation of volumetric data sets. In contrast, most radiology tests still contain only 2D images. We introduced a new testing tool that allows for stack viewing of volumetric images in our undergraduate radiology program. We hypothesized that tests with volumetric CT-images enhance test quality, in comparison with traditional completely 2D image-based tests, because they might better reflect required skills for clinical practice.

MATERIALS AND METHODS

Two groups of medical students (n=139; n=143), trained with 2D and volumetric CT-images, took a digital radiology test in two versions (A and B), each containing both 2D and volumetric CT-image questions. In a questionnaire, they were asked to comment on the representativeness for clinical practice, difficulty and user-friendliness of the test questions and testing program. Students' test scores and reliabilities, measured with Cronbach's alpha, of 2D and volumetric CT-image tests were compared.

RESULTS

Estimated reliabilities (Cronbach's alphas) were higher for volumetric CT-image scores (version A: .51 and version B: .54), than for 2D CT-image scores (version A: .24 and version B: .37). Participants found volumetric CT-image tests more representative of clinical practice, and considered them to be less difficult than volumetric CT-image questions. However, in one version (A), volumetric CT-image scores (M 80.9, SD 14.8) were significantly lower than 2D CT-image scores (M 88.4, SD 10.4) (p<.001). The volumetric CT-image testing program was considered user-friendly.

CONCLUSION

This study shows that volumetric image questions can be successfully integrated in students' radiology testing. Results suggests that the inclusion of volumetric CT-images might improve the quality of radiology tests by positively impacting perceived representativeness for clinical practice and increasing reliability of the test.