Bias in Radiology: The How and Why of Misses and Misinterpretations

Diagnostic Errors in Cerebrovascular Pathology: Retrospective Analysis of a Neuroradiology Database at a Large Tertiary Academic Medical Center

BACKGROUND AND PURPOSE

Diagnostic errors affect 2%-8% of neuroradiology studies, resulting in significant potential morbidity and mortality. This retrospective analysis of a large database at a single tertiary academic institution focuses on diagnostic misses in cerebrovascular pathology and suggests error-reduction strategies.

MATERIALS AND METHODS

CT and MR imaging reports from a consecutive database spanning 2015-2020 were searched for errors of attending physicians in cerebrovascular pathology. Data were collected on missed findings, study types, and interpretation settings. Errors were categorized as ischemic, arterial, venous, hemorrhagic, and "other."

RESULTS

A total of 245,762 CT and MR imaging neuroradiology examinations were interpreted during the study period. Vascular diagnostic errors were present in 165 reports, with a mean of 49.6 (SD, 23.3) studies on the shifts when an error was made, compared with 34.9 (SD, 19.2) on shifts without detected errors (P < .0001). Seventy percent of examinations occurred in the hospital setting; 93.3% of errors were perceptual; 6.7% were interpretive; and 93.9% (n = 155) were clinically significant (RADPEER 2B or 3B). The distribution of errors was arterial and ischemic each with 33.3%, hemorrhagic with 21.8%, and venous with 7.5%. Most errors involved brain MR imaging (30.3%) followed by head CTA (27.9%) and noncontrast head CT (26.1%). The most common misses were acute/subacute infarcts (25.1%), followed by aneurysms (13.7%) and subdural hematomas (9.7%).

CONCLUSIONS

Most cerebrovascular diagnostic errors were perceptual and clinically significant, occurred in the emergency/inpatient setting, and were associated with higher-volume shifts. Diagnostic errors could be minimized by adjusting search patterns to ensure vigilance on the sites of the frequently missed pathologies.

Histopathological diagnosis of cutaneous melanocytic lesions: blinded and nonblinded second opinions offer similar improvement in diagnostic accuracy

BACKGROUND

Previous studies of second opinions in the diagnosis of melanocytic skin lesions have examined blinded second opinions, which do not reflect usual clinical practice. The current study, conducted in the USA, investigated both blinded and nonblinded second opinions for their impact on diagnostic accuracy.

METHODS

In total, 100 melanocytic skin biopsy cases, ranging from benign to invasive melanoma, were interpreted by 74 dermatopathologists. Subsequently, 151 dermatopathologists performed nonblinded second and third reviews. We compared the accuracy of single reviewers, second opinions obtained from independent, blinded reviewers and second opinions obtained from sequential, nonblinded reviewers. Accuracy was defined with respect to a consensus reference diagnosis.

RESULTS

The mean case-level diagnostic accuracy of single reviewers was 65.3% (95% CI 63.4-67.2%). Second opinions arising from sequential, nonblinded reviewers significantly improved accuracy to 69.9% (95% CI 68.0-71.7%; P < 0.001). Similarly, second opinions arising from blinded reviewers improved upon the accuracy of single reviewers (69.2%; 95% CI 68.0-71.7%). Nonblinded reviewers were more likely than blinded reviewers to give diagnoses in the same diagnostic classes as the first diagnosis. Nonblinded reviewers tended to be more confident when they agreed with previous reviewers, even with inaccurate diagnoses.

CONCLUSION

We found that both blinded and nonblinded second reviewers offered a similar modest improvement in diagnostic accuracy compared with single reviewers. Obtaining second opinions with knowledge of previous reviews tends to generate agreement among reviews, and may generate unwarranted confidence in an inaccurate diagnosis. Combining aspects of both blinded and nonblinded review in practice may leverage the advantages while mitigating the disadvantages of each approach. Specifically, a second pathologist could give an initial diagnosis blinded to the results of the first pathologist, with subsequent nonblinded discussion between the two pathologists if their diagnoses differ.

Possible Bias in Supervised Deep Learning Algorithms for CT Lung Nodule Detection and Classification

Simple Summary

Artificial Intelligence (AI) algorithms can assist clinicians in their daily tasks by automatically detecting and/or classifying nodules in chest CT scans. Bias of such algorithms is one of the reasons why implementation of them in clinical practice is still not widely adopted. There is no published review on the bias that these algorithms may contain. This review aims to present different types of bias in such algorithms and present possible ways to mitigate them. Only then it would be possible to ensure that these algorithms work as intended under many different clinical settings.

Abstract

Artificial Intelligence (AI) algorithms for automatic lung nodule detection and classification can assist radiologists in their daily routine of chest CT evaluation. Even though many AI algorithms for these tasks have already been developed, their implementation in the clinical workflow is still largely lacking. Apart from the significant number of false-positive findings, one of the reasons for that is the bias that these algorithms may contain. In this review, different types of biases that may exist in chest CT AI nodule detection and classification algorithms are listed and discussed. Examples from the literature in which each type of bias occurs are presented, along with ways to mitigate these biases. Different types of biases can occur in chest CT AI algorithms for lung nodule detection and classification. Mitigation of them can be very difficult, if not impossible to achieve completely.

Influence of Prior Imaging Information on Diagnostic Accuracy for Focal Skeletal Processes—A Retrospective Analysis of the Consistency between Biopsy-Verified Imaging Diagnoses

Introduction: Comparing imaging examinations with those previously obtained is considered mandatory in imaging guidelines. To our knowledge, no studies are available on neither the influence, nor the sequence, of prior imaging and reports on diagnostic accuracy using biopsy as the reference standard. Such data are important to minimize diagnostic errors and to improve the preparation of diagnostic imaging guidelines. The aim of our study was to provide such data. Materials and methods: A retrospective cohort of 216 consecutive skeletal biopsies from patients with at least 2 different imaging modalities (X-ray, CT and MRI) performed within 6 months of biopsy was identified. The diagnostic accuracy of the individual imaging modality was assessed. Finally, the possible influence of the sequence of imaging modalities was investigated. Results: No significant difference in the accuracy of the imaging modalities was shown, being preceded by another imaging modality or not. However, the sequence analyses indicate sequential biases, particularly if MRI was the first imaging modality. Conclusion: The sequence of the imaging modalities seems to influence the diagnostic accuracy against a pathology reference standard. Further studies are needed to establish evidence-based guidelines for the strategy of using previous imaging and reports to improve diagnostic accuracy.

MRA-detected intracranial atherosclerotic disease in patients with TIA and minor stroke

OBJECTIVES

Patients with TIA and minor stroke commonly undergo CT and CTA in the emergency department with subsequent MRI with MRA for further workup. The purpose of this study was to review outpatient MRIs for TIA/stroke patients to assess the additional benefit, if any, of the MRA sequence in the detection of intracranial atherosclerotic disease in patients for whom CTA had already been performed.

METHODS

The radiology reports of outpatient MRIs of the brain for TIA/minor stroke patients were retrospectively reviewed via the hospital PACS system. Following this, the imaging report from the patient's initial presentation to the emergency department was reviewed. This index imaging and subsequent MRI were compared to assess the incidence of new vascular findings detected on the MRA sequence in patients for whom CTA had already been performed. Where new lesions had been identified at follow-up, the imaging was retroactively reviewed to assess if they were present on the index imaging.

RESULTS

Two hundred seven consecutive patients were reviewed. Significant (> 50%) intracranial atherosclerotic disease was present on MRA in 18 patients (8.7%). This was a new finding in 11 patients. Five had initial CTA where the atherosclerosis was not detected. All 5 of these cases were located in the posterior cerebral arteries. Incidental aneurysms were seen in 14 (6.7%); 12 were a new finding at time of MRI.

CONCLUSION

The MRA sequence provides additional value by increasing the detection of clinically important intracranial atherosclerotic disease which may inform management in patients with minor stroke and TIA.

Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists

RATIONALE AND OBJECTIVES

Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers.

MATERIALS AND METHODS

This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1).

RESULTS

Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer.

CONCLUSION

Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

Impact of explainable artificial intelligence assistance on clinical decision-making of novice dental clinicians

Objective

Despite artificial intelligence (AI) being used increasingly in healthcare, implementation challenges exist leading to potential biases during the clinical decision process of the practitioner. The interaction of AI with novice clinicians was investigated through an identification task, an important component of diagnosis, in dental radiography. The study evaluated the performance, efficiency, and confidence level of dental students on radiographic identification of furcation involvement (FI), with and without AI assistance.

Materials and Methods

Twenty-two third- and 19 fourth-year dental students (DS3 and DS4, respectively) completed remotely administered surveys to identify FI lesions on a series of dental radiographs. The control group received radiographs without AI assistance while the test group received the same radiographs and AI-labeled radiographs. Data were appropriately analyzed using the Chi-square, Fischer's exact, analysis of variance, or Kruskal-Wallis tests.

Results

Performance between groups with and without AI assistance was not statistically significant except for 1 question where tendency was to err with AI-generated answer (P < .05). The efficiency of task completion and confidence levels was not statistically significant between groups. However, both groups with and without AI assistance believed the use of AI would improve the clinical decision-making.

Discussion

Dental students detecting FI in radiographs with AI assistance had a tendency towards over-reliance on AI.

Conclusion

AI input impacts clinical decision-making, which might be particularly exaggerated in novice clinicians. As it is integrated into routine clinical practice, caution must be taken to prevent overreliance on AI-generated information.

Enhancing medical-imaging artificial intelligence through holistic use of time-tested key imaging and clinical parameters: Future insights

Much of the published literature in Radiology-related Artificial Intelligence (AI) focuses on single tasks, such as identifying the presence or absence or severity of specific lesions. Progress comparable to that achieved for general-purpose computer vision has been hampered by the unavailability of large and diverse radiology datasets containing different types of lesions with possibly multiple kinds of abnormalities in the same image. Also, since a diagnosis is rarely achieved through an image alone, radiology AI must be able to employ diverse strategies that consider all available evidence, not just imaging information. Using key imaging and clinical signs will help improve their accuracy and utility tremendously. Employing strategies that consider all available evidence will be a formidable task; we believe that the combination of human and computer intelligence will be superior to either one alone. Further, unless an AI application is explainable, radiologists will not trust it to be either reliable or bias-free; we discuss some approaches aimed at providing better explanations, as well as regulatory concerns regarding explainability (“transparency”). Finally, we look at federated learning, which allows pooling data from multiple locales while maintaining data privacy to create more generalizable and reliable models, and quantum computing, still prototypical but potentially revolutionary in its computing impact.

Quality-assured training in the evaluation of cochlear implant electrode position: a prospective experimental study

BACKGROUND

The objective of this study was to demonstrate the utility of an approach in training predoctoral medical students, to enable them to measure electrode-to-modiolus distances (EMDs) and insertion-depth angles (aDOIs) in cochlear implant (CI) imaging at the performance level of a single senior rater.

METHODS

This prospective experimental study was conducted on a clinical training dataset comprising patients undergoing cochlear implantation with a Nucleus® CI532 Slim Modiolar electrode (N = 20) or a CI512 Contour Advance electrode (N = 10). To assess the learning curves of a single medical student in measuring EMD and aDOI, interrater differences (senior-student) were compared with the intrarater differences of a single senior rater (test-retest). The interrater and intrarater range were both calculated as the distance between the 0.1th and 99.9th percentiles. A "deliberate practice" training approach was used to teach knowledge and skills, while correctives were applied to minimize faulty data-gathering and data synthesis.

RESULTS

Intrarater differences of the senior rater ranged from - 0.5 to 0.5 mm for EMD and - 14° to 16° for aDOI (respective medians: 0 mm and 0°). Use of the training approach led to interrater differences that matched this after the 4th (EMD) and 3rd (aDOI) feedback/measurement series had been provided to the student.

CONCLUSIONS

The training approach enabled the student to evaluate the CI electrode position at the performance level of a senior rater. This finding may offer a basis for ongoing clinical quality assurance for the assessment of CI electrode position.

DECOMP Report: Answers surgeons expect from an abdominal wall imaging exam

Abdominal wall (AW) hernias are a common problem faced by general surgeons. With an essentially clinical diagnosis, abdominal hernias have been considered a simple problem to be repaired. However, long-term follow-up of patients has shown disappointing results, both in terms of complications and recurrence. In this context, preoperative planning with control of comorbidities and full knowledge of the hernia and its anatomical relationships with the AW has gained increasing attention. Computed tomography (CT) appears to be the best option to determine the precise size and location of abdominal hernias, presence of rectus diastase and/or associated muscle atrophy, as well as the proportion of the hernia in relation to the AW itself. This information might help the surgeon to choose the best surgical technique (open vs MIS), positioning and fixation of the meshes, and eventual need for application of botulinum toxin, preoperative pneumoperitoneum or component separation techniques. Despite the relevance of the findings, they are rarely described in CT scans as radiologists are not used to report findings of the AW as well as to know what information is really needed. For these reasons, we gathered a group of surgeons and radiologists to establish which information about the AW is important in a CT. Finally, a structured report is proposed to facilitate the description of the findings and their interpretation.

Cognitive Bias and Diagnostic Errors among Physicians in Japan: A Self-Reflection Survey

This cross-sectional study aimed to clarify how cognitive biases and situational factors related to diagnostic errors among physicians. A self-reflection questionnaire survey on physicians’ most memorable diagnostic error cases was conducted at seven conferences: one each in Okayama, Hiroshima, Matsue, Izumo City, and Osaka, and two in Tokyo. Among the 147 recruited participants, 130 completed and returned the questionnaires. We recruited primary care physicians working in various specialty areas and settings (e.g., clinics and hospitals). Results indicated that the emergency department was the most common setting (47.7%), and the highest frequency of errors occurred during night-time work. An average of 3.08 cognitive biases was attributed to each error. The participants reported anchoring bias (60.0%), premature closure (58.5%), availability bias (46.2%), and hassle bias (33.1%), with the first three being most frequent. Further, multivariate logistic regression analysis for cognitive bias showed that emergency room care can easily induce cognitive bias (adjusted odds ratio 3.96, 95% CI 1.16−13.6, p-value = 0.028). Although limited to a certain extent by its sample collection, due to the sensitive nature of information regarding physicians’ diagnostic errors, this study nonetheless shows correlations with environmental factors (emergency room care situations) that induce cognitive biases which, in turn, cause diagnostic errors.

Hiding in Plain Sight: A Retrospective Review of Unrecognized Tumors During Dermatologic Surgery

Background: Mohs micrographic surgery requires focused attention that may lead to tunnel vision bias, contributing to not recognizing skin cancer at nearby sites.

Objective: It is to determine if a subsequently diagnosed skin cancer was visible at the time of Mohs surgery.

Methods: A retrospective chart review was performed at a single academic center from 2008 to 2020. Patients who underwent at least two distinct MMS procedures, separated in time to capture subsequent tumors, were included.

Results: Four hundred and four individual patients were identified with at least two distinct Mohs procedures, which generated 1,110 Mohs sequences. Fifty-one (4.6%) clinically apparent tumors went unrecognized and 127 (11.4%) tumors were identified and biopsied during the visit. High-risk tumor histology was identified in 10 (20%) unrecognized tumors and 31 (24%) recognized tumors (p-value 0.491).

Conclusion: Our study suggests that Mohs surgeons may be overlooking adjacent skin cancers when focusing only on the tumor being surgically treated. Tunnel vision bias may account for part of this phenomenon.

Diagnostic Imaging and Mechanical Objectivity in Medicine

BACKGROUND

Before the advent of automatism in image-making practices, scientists, anatomists, and physicians artistically depicted simplified images for scientific atlas making. This technique conferred subjectivity to a supposedly objective scientific process, sparking confrontations between anatomists regarding accuracy that heralded a new concept in the late 19^th century - mechanical objectivity - that would revolutionize scientific knowledge and the field of medicine OBJECTIVES: The purpose of this health history research study is to trace the evolution of mechanical objectivity from empirical studies of early anatomists in the 19^th century to the advent of x-ray technology, digitization of imaging, and disruptive technological innovations such as artificial intelligence, while simultaneously unveiling the challenges of mitigating human bias, despite advancements in medical imaging practices.

METHODS

This narrative literature review was conducted using the Scopus® database under the guidance of both medical historians and practicing physicians to ensure its applicability and historical accuracy CONCLUSION: Despite a century-long quest for optimizing mechanical objectivity in diagnostic imaging to more accurately and efficiently interpret medical images, human bias remains an important factor. This historical review describes the development of medical imaging technologies over the last century with emphasis on the role played by human bias and subjectivity in a rapidly expanding field of medical imaging technology including artificial intelligence.

Lessons Learned from Peer Learning Conference in Cardiothoracic Radiology

Medical errors may lead to patient harm and may also have a devastating effect on medical providers, who may suffer from guilt and the personal impact of a given error (second victim experience). While it is important to recognize and remedy errors, it should be done in a way that leads to long-standing practice improvement and focuses on systems-level opportunities rather than in a punitive fashion. Traditional peer review systems are score based and have some undesirable attributes. The authors discuss the differences between traditional peer review systems and peer learning approaches and offer practical suggestions for transitioning to peer learning conferences. Peer learning conferences focus on learning opportunities and embrace errors as an opportunity to learn. The authors also discuss various types and sources of errors relevant to the practice of radiology and how discussions in peer learning conferences can lead to widespread system improvement. In the authors' experience, these strategies have resulted in practice improvement not only at a division level in radiology but in a broader multidisciplinary setting as well. The online slide presentation from the RSNA Annual Meeting is available for this article. ^©RSNA, 2022.

Improving the Relationship Between Confidence and Competence: Implications for Diagnostic Radiology Training From the Psychology and Medical Literature

The focus of diagnostic radiology training is on creating competent professionals, whereas confidence and its calibration receive less attention. Appropriate confidence is critical for patient care both during and after training. Overconfidence can adversely affect patient care and underconfidence can create excessive costs. We reviewed the psychology and medical literature pertaining to confidence and competence to collect insights and best practices from the psychology and medical literature on confidence and apply them to radiology training. People are rarely accurate in assessments of their own competence. Among physicians, the correlation between perceived abilities and external assessments of those abilities is weak. Overconfidence is more prevalent than underconfidence, particularly at lower levels of competence. On the individual level, confidence can be calibrated to a more appropriate level through efforts to increase competence, including sub-specialization, and by gaining a better understanding of metacognitive processes. With feedback, high-fidelity simulation has the potential to improve both competence and metacognition. On the system level, systems that facilitate access to follow-up imaging, pathology, and clinical outcomes can help close the gap between perceived and actual performance. Appropriate matching of trainee confidence and competence should be a goal of radiology residency and fellowship training to help mitigate the adverse effects of both overconfidence and underconfidence during training and independent practice.

Giant duodenal diverticulum masquerading as a sealed perforation

Giant duodenal diverticula are large outpouchings involving all layers of the duodenal wall. Whilst often an incidental finding, giant duodenal diverticula can present with diverticulitis or biliary obstruction. We report a case of a giant duodenal diverticulum that was initially misdiagnosed as a localised duodenal perforation on CT. Additional ultrasound and fluoroscopic imaging demonstrated the final diagnosis of acute cholecystitis. The clinical course of this patient highlights the challenge of recognising a giant duodenal diverticulum and the limitations of solely relying on CT in the context of an acute abdominal presentation.

Deep learning detection of subtle fractures using staged algorithms to mimic radiologist search pattern

OBJECTIVE

To develop and evaluate a two-stage deep convolutional neural network system that mimics a radiologist's search pattern for detecting two small fractures: triquetral avulsion fractures and Segond fractures.

MATERIALS AND METHODS

We obtained 231 lateral wrist radiographs and 173 anteroposterior knee radiographs from the Stanford MURA and LERA datasets and the public domain to train and validate a two-stage deep convolutional neural network system: (1) object detectors that crop the dorsal triquetrum or lateral tibial condyle, trained on control images, followed by (2) classifiers for triquetral and Segond fractures, trained on a 1:1 case:control split. A second set of classifiers was trained on uncropped images for comparison. External test sets of 50 lateral wrist radiographs and 24 anteroposterior knee radiographs were used to evaluate generalizability. Gradient-class activation mapping was used to inspect image regions of greater importance in deciding the final classification.

RESULTS

The object detectors accurately cropped the regions of interest in all validation and test images. The two-stage system achieved cross-validated area under the receiver operating characteristic curve values of 0.959 and 0.989 on triquetral and Segond fractures, compared with 0.860 (p = 0.0086) and 0.909 (p = 0.0074), respectively, for a one-stage classifier. Two-stage cross-validation accuracies were 90.8% and 92.5% for triquetral and Segond fractures, respectively.

CONCLUSION

A two-stage pipeline increases accuracy in the detection of subtle fractures on radiographs compared with a one-stage classifier and generalized well to external test data. Focusing attention on specific image regions appears to improve detection of subtle findings that may otherwise be missed.

The Effect of Clinical History on Diagnostic Imaging Interpretation - A Systematic Review

RATIONALE AND OBJECTIVES

To provide updated information on the effect of clinical history on diagnostic image interpretation and to provide study methodology and design recommendations for future studies assessing the effect of clinical history on diagnostic image performance.

MATERIALS AND METHODS

A literature search of Medline, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials (CENTRAL) databases was conducted from database inception to July 21, 2020. Studies comparing diagnostic imaging performance with and without clinical history, using observers reading images under both conditions that used an independent reference standard were included.

RESULTS

Twenty-two studies met the inclusion criteria, with 15 showing clinical history improved diagnostic performance. One study reported a decrease in diagnostic performance with clinical history and the remaining six studies found no significant change in performance. Two studies used the free response paradigm with both reporting clinical history increased location sensitivity, decreased specificity and had no overall change in diagnostic performance. The disease spectrum of included cases was largely unreported and a balanced reading design was not used in 19 studies.

CONCLUSION

Most published studies found that clinical history improved diagnostic performance. More recent studies accounting for abnormality location and multiple abnormalities showed an increase in false positives and no significant change in overall diagnostic performance with clinical history.

CAR Peer Learning Guide

Peer learning is a quality initiative used to identify potential areas of practice improvement, both on a patient level and on a systemic level. Opportunities for peer learning include review of prior imaging studies, evaluation of cases from multidisciplinary case conferences, and review of radiology trainees' call cases. Peer learning is non-punitive and focuses on promoting life-long learning. It seeks to identify and disseminate learning opportunities and areas for systems improvement compared to traditional peer review. Learning opportunities arise from peer learning through both individual communication of cases reviewed for routine work, as well as through anonymous presentation of aggregate cases in an educational format. In conjunction with other tools such as root cause analysis, peer learning can be used to guide future practice improvement opportunities. This guide provides definitions of terms and a synthetic evidence review regarding peer review and peer learning, as well as medicolegal and jurisdictional considerations. Important aspects of what makes an effective peer learning program and best practices for implementing such a program are presented. The guide is intended to be a living document that will be updated regularly as new data emerges and peer learning continues to evolve in radiology practices.

Relatório DECOMP: Respostas que os cirurgiões esperam de um exame de imagem da parede abdominal

RESUMO Hérnias da parede abdominal são um problema bastante comum enfrentado pelo cirurgiões gerais. De diagnóstico essencialmente clínico, as hérnias abdominais durante muito tempo têm sido consideradas um problema de simples reparo. Entretanto, o acompanhamento de longo prazo dos pacientes têm demonstrado resultados desapontadores, tanto em termos de complicações quanto risco de recidiva da hérnia. Neste contexto, o planejamento pré-operatório com controle de comorbidades e pleno conhecimento da hérnia e suas relações anatômicas com a parede abdominal têm ganho cada vez mais atenção. A tomografia de abdome parece ser a melhor opção para determinar o tamanho e localização precisos das hérnias abdominais, presença de diastase de músculo reto e/ou atrofia da parede associada, assim como proporção da hérnia em relação a parede abdominal. Essas informações podem auxiliar o cirurgião na escolha da melhor técnica cirúrgica (aberta vs. MIS), posicionamento e fixação das telas, e eventual necessidade de aplicação de toxina botulínica, pneumoperitônio pré-operatório ou técnicas de separação de componentes. Apesar da relevância dos achados, eles são raramente descritos em exames de tomografia uma vez que os radiologistas não estão acostumados a olhar para a parede abdominal assim como não sabem quais as informações são realmente necessárias. Por estes motivos, nós reunimos um grupo de cirurgiões e radiologistas visando estabelecer quais são as informações da parede abdominal mais importantes em um exame de tomografia assim como propor um laudo estruturado para facilitar a descrição dos achados e sua interpretação.

Social Asymmetry, Artificial Intelligence and the Medical Imaging Landscape

Social and health care equity and justice should be prioritized by the mantra of medicine, first do no harm. Despite highly motivated national and global health strategies, there remains significant health care inequity. Intrinsic and extrinsic factors, including a number of biases, are key drivers of ongoing health inequity including equity of access and opportunity for nuclear medicine and radiology services. There is a substantial gap in the global practice of nuclear medicine in particular, but also radiology, between developed health economies and those considered developing or undeveloped. At a local level, even in developed health economies, there can be a significant disparity between health services, including medical imaging, between communities based on socioeconomic, cultural or geographic differences. Artificial intelligence (AI) has the potential to either widen the health inequity divide or substantially reduce it. Distributed generally, AI technology could be used to overcome geographic boundaries to health care, thus bringing general and specialist care into underserved communities. However, should AI technology be limited to localities already enjoying ample healthcare access and direct access to health infrastructure, like radiology and nuclear medicine, it could then accentuate the gap. There are a number of challenges across the AI pipeline that need careful attention to ensure beneficence over maleficence. Fully realized, AI augmented health care could be crafted as an integral part of the broader strategy convergence on local, national and global health equity. The applications of AI in nuclear medicine and radiology could emerge as a powerful tool in social and health equity.

CapsCovNet: A Modified Capsule Network to Diagnose COVID-19 From Multimodal Medical Imaging

Since the end of 2019, novel coronavirus disease (COVID-19) has brought about a plethora of unforeseen changes to the world as we know it. Despite our ceaseless fight against it, COVID-19 has claimed millions of lives, and the death toll exacerbated due to its extremely contagious and fast-spreading nature. To control the spread of this highly contagious disease, a rapid and accurate diagnosis can play a very crucial part. Motivated by this context, a parallelly concatenated convolutional block-based capsule network is proposed in this article as an efficient tool to diagnose the COVID-19 patients from multimodal medical images. Concatenation of deep convolutional blocks of different filter sizes allows us to integrate discriminative spatial features by simultaneously changing the receptive field and enhances the scalability of the model. Moreover, concatenation of capsule layers strengthens the model to learn more complex representation by presenting the information in a fine to coarser manner. The proposed model is evaluated on three benchmark datasets, in which two of them are chest radiograph datasets and the rest is an ultrasound imaging dataset. The architecture that we have proposed through extensive analysis and reasoning achieved outstanding performance in COVID-19 detection task, which signifies the potentiality of the proposed model.

The Added Effect of Artificial Intelligence on Physicians' Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review

Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.

The CT scout view: complementary value added to abdominal CT interpretation

Computed tomography (CT) scout images, also known as CT localizer radiographs, topograms, or scanograms, are an important, albeit often overlooked part of the CT examination. Scout images may contain important findings outside of the scanned field of view on CT examinations of the abdomen and pelvis, such as unsuspected lung cancer at the lung bases. Alternatively, scout images can provide complementary information to findings within the scanned field of view, such as characterization of retained surgical foreign bodies. Assessment of scout images adds value and provides a complementary "opportunistic" review for interpretation of abdominopelvic CT examinations. Scout image review is a useful modern application of conventional abdominal radiograph interpretation that can help establish a diagnosis or narrow a differential diagnosis. This review discusses the primary purpose and intent of the CT scout images, addresses standard of care and bias related to scout image review, and presents a general systematic approach to assessing scout images with multiple illustrative examples, including potential pitfalls in interpreting scout images.

Technical and interpretive pitfalls of postmortem CT: Five examples of errors revealed by autopsy

Image acquisition of dead bodies, particularly using postmortem computed tomography (PMCT), has become common in forensic investigations worldwide. Meanwhile, in countries such as Japan which have an extremely low rate of autopsy, PMCT is being increasingly used in the clinical field to certify the cause of death (COD) without performing an autopsy or toxicological tests, even in cases of unnatural death. Additionally, these PMCT images are predominantly interpreted by clinical personnel such as emergency physicians or clinicians who are not trained in PMCT interpretation and who work for the police, that is, the so-called police doctors. Many potential pitfalls associated with the use of PMCT have been previously described in textbooks and published papers, including the pitfalls of not performing a complete forensic pathology investigation, and the use of physicians without appropriate PMCT training to interpret PMCT and direct death investigation and certification. We describe five examples in which apparent misdiagnosis of COD based on PMCT misinterpretation was revealed by autopsy. Here are the five examples of errors: (1) Postmortem changes were misinterpreted as COD, (2) resuscitation effects were misinterpreted as COD, (3) COD was determined after an incomplete examination, (4) fatal findings caused by external origin were wrongly interpreted as 'of internal origin' based on PMCT, and (5) non-fatal findings on PMCT were wrongly interpreted as fatal. Interpretation of PMCT by appropriately trained physicians and an accompanying complete forensic investigation, including autopsy when indicated, is necessary to prevent significant errors in COD determination and related potential adverse medicolegal consequences.

Error and cognitive bias in diagnostic radiology

The above article was posted prematurely on 31 August 2021. The article will be made fully available at a later date.

Brain MRI Deep Learning and Bayesian Inference System Augments Radiology Resident Performance

Automated quantitative and probabilistic medical image analysis has the potential to improve the accuracy and efficiency of the radiology workflow. We sought to determine whether AI systems for brain MRI diagnosis could be used as a clinical decision support tool to augment radiologist performance. We utilized previously developed AI systems that combine convolutional neural networks and expert-derived Bayesian networks to distinguish among 50 diagnostic entities on multimodal brain MRIs. We tested whether these systems could augment radiologist performance through an interactive clinical decision support tool known as Adaptive Radiology Interpretation and Education System (ARIES) in 194 test cases. Four radiology residents and three academic neuroradiologists viewed half of the cases unassisted and half with the results of the AI system displayed on ARIES. Diagnostic accuracy of radiologists for top diagnosis (TDx) and top three differential diagnosis (T3DDx) was compared with and without ARIES. Radiology resident performance was significantly better with ARIES for both TDx (55% vs 30%; P < .001) and T3DDx (79% vs 52%; P = 0.002), with the largest improvement for rare diseases (39% increase for T3DDx; P < 0.001). There was no significant difference between attending performance with and without ARIES for TDx (72% vs 69%; P = 0.48) or T3DDx (86% vs 89%; P = 0.39). These findings suggest that a hybrid deep learning and Bayesian inference clinical decision support system has the potential to augment diagnostic accuracy of non-specialists to approach the level of subspecialists for a large array of diseases on brain MRI.

Chest radiographs and machine learning - Past, present and future

Despite its simple acquisition technique, the chest X-ray remains the most common first-line imaging tool for chest assessment globally. Recent evidence for image analysis using modern machine learning points to possible improvements in both the efficiency and the accuracy of chest X-ray interpretation. While promising, these machine learning algorithms have not provided comprehensive assessment of findings in an image and do not account for clinical history or other relevant clinical information. However, the rapid evolution in technology and evidence base for its use suggests that the next generation of comprehensive, well-tested machine learning algorithms will be a revolution akin to early advances in X-ray technology. Current use cases, strengths, limitations and applications of chest X-ray machine learning systems are discussed.

Diagnostic Errors in Neuroradiology: A Message to Emergency Radiologists and Trainees

Diagnostic errors in neuroradiology are inevitable, yet potentially avoidable. Through extensive literature search, we present an up-to-date review of the psychology of human decision making and how such complex process can lead to radiologic errors. Our focus is on neuroradiology, so we augmented our review with multiple explanatory figures to show how different errors can reflect on real-life clinical practice. We propose a new thematic categorization of perceptual and cognitive biases in this article to simplify message delivery to our target audience: emergency/general radiologists and trainees. Additionally, we highlight individual and organizational remedy strategies to decrease error rate and potential harm.

Analysis of Literature Regarding Health Care Disparities in Radiology: Is Radiology Falling Behind?

PURPOSE

To quantify the gap between radiology and other specialties regarding the amount of literature on healthcare disparities (HCD).

METHODS

Four different searches were conducted to evaluate the amount of literature on HCD in radiology as compared to internal medicine and surgery. Initially, the Journal Citation Report was utilized to search for the five highest ranking journals in each field and a second search used impact factor. A combination of search terms "health" AND "disparities" was used. Two additional searches were performed with PubMed using the terms "health" AND "disparities AND "radiology" with the final term changed for each specialty. The second PubMed search added the term "medical education" for each specialty. Articles were limited to years 2017 to 2020.

RESULTS

The initial search found 1817 articles discussing "health" and "disparities". 14.6% of these were radiology, 65.7% internal medicine, and 19.7% surgery. The subsequent search controlling for impact factor found 2176 articles. 12.2% were for radiology, 66.1% were for internal medicine, and 21.7% for surgery. The initial PubMed search found 6543 articles. 9.9% were for radiology, 32.4% for internal medicine, and 57.7% were for surgery. The addition of "medical education" decreased the articles to 807. Radiology had 9.9%, internal medicine was 44.2%, and surgery was 45.9 %.

CONCLUSION

A gap in HCD literature exists for radiology as compared to surgery and internal medicine. However, radiology has demonstrated a recent significant push towards understanding HCD.  Radiology should continue to capitalize on its momentum and develop HCD curricula and research.

Medical Student Elective to Improve Diagnosis in Health Care: Developing Solutions to Reduce Patient Harm

RATIONALE AND OBJECTIVES

At our institution, a new medical student elective was designed and implemented by the department of radiology to teach medical students about diagnostic error. The purpose of this article is to describe the diagnostic error elective structure and implementation, present objective and subjective evaluations of the elective, and provide a model for other radiology departments to run their own electives.

MATERIALS AND METHODS

Starting in January 2018, a 2-week in-person career exploration session elective was offered for third year medical students. In 2020 due to the COVID pandemic, the elective was expanded to fourth year medical students. All students were required to complete a project that addressed diagnostic error. Subjective comments were recorded, and objective measurements obtained from student evaluations.

RESULTS

A total of 11 sessions were held, consisting of 3 fourth year and 26 third year students. A total of 12 projects (11 groups) were completed, seven of which have been accepted for presentation at national meetings. On a 1 to 5 scale (5 highest), students rated their educational experience at a mean score of 4.61. Subjective comments focused on the benefit of exposure to new topics, mentorship by radiologists, and ability to complete a project in such a short time.

CONCLUSION

Diagnostic errors and solutions are vague, new concepts to medical students and even facilitator faculty. This course allowed students to gain awareness of diagnostic error and could easily be replicated at other institutions with interested faculty and medical school support.

Investigating errors in medical imaging: medical malpractice cases in Finland

Objective

The objectives of the study were to survey patient injury claims concerning medical imaging in Finland in 1991–2017, and to investigate the nature of the incidents, the number of claims, the reasons for the claims, and the decisions made concerning the claims.

Materials and methods

The research material consisted of patient claims concerning imaging, sent to the Finnish Patient Insurance Centre (PVK). The data contained information on injury dates, the examination code, the decision code, the description of the injury, and the medical grounds for decisions.

Results

The number of claims included in the study was 1054, and the average number per year was 87. The most common cause was delayed diagnosis (404 claims, 38.3%). Most of the claims concerned mammography (314, 29.8%), radiography (170, 16.1%), and MRI (162, 15.4%). According to the decisions made by the PVK, there were no delays in 54.6% of the examinations for which claims were made. About 30% of all patient claims received compensation, the most typical reason being medical malpractice (27.7%), followed by excessive injuries and injuries caused by infections, accidents and equipment (2.7%).

Conclusion

Patient injury in imaging examinations and interventions cannot be completely prevented. However, injury data are an important source of information for health care. By analysing claims, we can prevent harm, increase the quality of care, and improve patient safety in medical imaging.

Direct Access and Skill Mix Can Reduce Telephone Interruptions and Imaging Wait Times: Improving Radiology Service Effectiveness, Safety and Sustainability

Unnecessary telephone calls to reporting radiologists impede organizations' workflow and may be associated with a higher chance of errors in reports. We conducted a prospective study in two cycles, which identified vetting plain CT heads as the most common reason for these calls and vetting CT urinary tracts (KUB) was also frequent. Clear vetting and protocolling guidelines exist for both of these scans, which do not routinely require discussion with a radiologist. Therefore, our approach was to create new flow diagrams to allow radiographers to directly accept routine requests for plain CT head and CT KUB scans in- and out-of-hours. After this intervention, incoming calls to radiology for vetting CT heads decreased by 30% and for vetting CT KUBs by 100%. The average wait time between CT head request and scan completion was reduced by 40%. The number of CT head and CT KUB scans performed remained stable. In future, maximizing the benefit of direct access in-patient imaging pathways will rely on effective and sustained communication of the protocols to the junior clinical staff rotating through the organization, as they were responsible for requesting the vast majority of tests.

Optimising breast cancer screening reading: blinding the second reader to the first reader's decisions

Objectives

In breast cancer screening, two readers separately examine each woman’s mammograms for signs of cancer. We examined whether preventing the two readers from seeing each other’s decisions (blinding) affects behaviour and outcomes.

Methods

This cohort study used data from the CO-OPS breast-screening trial (1,119,191 women from 43 screening centres in England) where all discrepant readings were arbitrated. Multilevel models were fitted using Markov chain Monte Carlo to measure whether reader 2 conformed to the decisions of reader 1 when they were not blinded, and the effect of blinding on overall rates of recall for further tests and cancer detection. Differences in positive predictive value (PPV) were assessed using Pearson’s chi-squared test.

Results

When reader 1 recalls, the probability of reader 2 also recalling was higher when not blinded than when blinded, suggesting readers may be influenced by the other’s decision. Overall, women were less likely to be recalled when reader 2 was blinded (OR 0.923; 95% credible interval 0.864, 0.986), with no clear pattern in cancer detection rate (OR 1.029; 95% credible interval 0.970, 1.089; Bayesian p value 0.832). PPV was 22.1% for blinded versus 20.6% for not blinded (p < 0.001).

Conclusions

Our results suggest that when not blinded, reader 2 is influenced by reader 1’s decisions to recall (alliterative bias) which would result in bypassing arbitration and negate some of the benefits of double-reading. We found a relationship between blinding the second reader and slightly higher PPV of breast cancer screening, although this analysis may be confounded by other centre characteristics.

Key Points

• In Europe, it is recommended that breast screening mammograms are analysed by two readers but there is little evidence on the effect of ‘blinding’ the readers so they cannot see each other’s decisions.

• We found evidence that when the second reader is not blinded, they are more likely to agree with a recall decision from the first reader and less likely to make an independent judgement (alliterative error). This may reduce overall accuracy through bypassing arbitration.

• This observational study suggests an association between blinding the second reader and higher positive predictive value of screening, but this may be confounded by centre characteristics.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-07965-z.

Giant retroperitoneal paraganglioma: Challenges of misdiagnosis and high surgical risks, a case report

Introduction and importance

In surgery, misdiagnosis is not uncommon, usually a result of erroneous image interpretations and pathology diagnosis especially involving a tumor or cancer. Misdiagnosis may cause increased morbidity, mortality and surgical risks.

Case presentation

A 49-year-old man presented for the second time with a right upper abdominal mass of 7 months. Previous CT scan of abdomen and exploratory surgery made the diagnosis of liver cancer. Two other tertiary hospitals drew the similar conclusions. At a cancer hospital the needle biopsy was suspicious for gastrointestinal stromal tumor, Imatinib was recommended but not started due to high cost. During this re-admission, the diagnosis of liver cancer or GIST was challenged. A high risk surgery was done with successive removal of a giant tumor. A final diagnosis of paraganlioma was made and the patient is now tumor free for 6 years.

Clinical discussion

There are 4 lessons from this case. First, a paraganlioma may be misdiagnosed. Second, the misdiagnosis may be misled by CT scan and pathology. Third, a misdiagnosis can cause increased morbidity, mortality and surgical risks. Forth, massive intraoperative hemorrhage is a high risk of surgery.

Conclusion

Careful clinical evaluation combined with pathology diagnosis may reduce the misdiagnosis of some tumor/cancer. Surgical resection may be the only way to reach a diagnosis in patient with paraganlioma. Massive intraoperative hemorrhage is a high risk of surgery in such patients.

•

Paraganglioma may be misdiagnosed as other tumor or cancer.

•

Computed tomography of abdomen is not diagnostic for paraganglioma.

•

Misdiagnosis can cause increased morbidity, mortality and surgical risks.

•

Careful clinical evaluation combined with pathological diagnosis can prevent or reduce misdiagnosis of abdominal solid mass.

•

Massive intraoperative hemorrhage is a high risk of surgery for large garaganglioma.

Ethical challenges in child abuse: what is the harm of a misdiagnosis?

In this article the author examines ethical tensions inherent to balancing harms of false-negative and false-positive child abuse diagnoses, and he describes how such tensions manifest in courtroom proceedings. Child abuse physicians, including pediatric radiologists, shoulder heavy responsibilities weighing the potential consequences of not diagnosing child abuse when it could have been diagnosed (false negatives) against the consequences of making the diagnosis when it has not occurred (false positives). These physicians, who practice under ethical obligations to serve children's best interests and protect them from harm, make daily practice decisions knowing that, on balance, abuse is substantially more underdiagnosed than over diagnosed. Legal justice advocates, however, emphasize that clinical decision-making around abuse is not disassociated from endemic injustices that unduly incriminate individuals from socioeconomically underprivileged populations. Some defense advocates charge that child abuse physicians are insufficiently sensitive to harms of erroneous diagnoses, and they have characterized these clinicians as frankly biased. To support their claims in court, defense advocates have enlisted likeminded physician witnesses whose credentials as experts flout professional standards and who provide consistently flawed testimony based upon deficiently peer-reviewed literature. This article concludes that, to help mitigate these unhealthy circumstances, child abuse physicians might build trust with criminal defense advocates by instituting measures to alleviate perceptions of biases and by more explicitly acknowledging the potential harms of erroneous diagnoses. Professional societies representing these physicians, such as the Society for Pediatric Radiology, could take concurrent measures to help better prepare their constituent clinicians for expert testimony and make them more available to testify.

Errors, discrepancies and underlying bias in radiology with case examples: a pictorial review

Interpretation differences between radiologists and diagnostic errors are significant issues in daily radiology practice. An awareness of errors and their underlying causes can potentially increase the diagnostic performance and reduce individual harm. The aim of this paper is to review both the classification of errors and the underlying biases. Case-based examples are presented and discussed for each type of error and bias to provide greater clarity and understanding.

An anomaly detection approach to identify chronic brain infarcts on MRI

The performance of current machine learning methods to detect heterogeneous pathology is limited by the quantity and quality of pathology in medical images. A possible solution is anomaly detection; an approach that can detect all abnormalities by learning how 'normal' tissue looks like. In this work, we propose an anomaly detection method using a neural network architecture for the detection of chronic brain infarcts on brain MR images. The neural network was trained to learn the visual appearance of normal appearing brains of 697 patients. We evaluated its performance on the detection of chronic brain infarcts in 225 patients, which were previously labeled. Our proposed method detected 374 chronic brain infarcts (68% of the total amount of brain infarcts) which represented 97.5% of the total infarct volume. Additionally, 26 new brain infarcts were identified that were originally missed by the radiologist during radiological reading. Our proposed method also detected white matter hyperintensities, anomalous calcifications, and imaging artefacts. This work shows that anomaly detection is a powerful approach for the detection of multiple brain abnormalities, and can potentially be used to improve the radiological workflow efficiency by guiding radiologists to brain anomalies which otherwise remain unnoticed.

Do as AI say: susceptibility in deployment of clinical decision-aids

Artificial intelligence (AI) models for decision support have been developed for clinical settings such as radiology, but little work evaluates the potential impact of such systems. In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not. Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.

Not all biases are bad: equitable and inequitable biases in machine learning and radiology

The application of machine learning (ML) technologies in medicine generally but also in radiology more specifically is hoped to improve clinical processes and the provision of healthcare. A central motivation in this regard is to advance patient treatment by reducing human error and increasing the accuracy of prognosis, diagnosis and therapy decisions. There is, however, also increasing awareness about bias in ML technologies and its potentially harmful consequences. Biases refer to systematic distortions of datasets, algorithms, or human decision making. These systematic distortions are understood to have negative effects on the quality of an outcome in terms of accuracy, fairness, or transparency. But biases are not only a technical problem that requires a technical solution. Because they often also have a social dimension, the 'distorted' outcomes they yield often have implications for equity. This paper assesses different types of biases that can emerge within applications of ML in radiology, and discusses in what cases such biases are problematic. Drawing upon theories of equity in healthcare, we argue that while some biases are harmful and should be acted upon, others might be unproblematic and even desirable-exactly because they can contribute to overcome inequities.

Absence of Posttraumatic Bone Marrow Edema in the Setting of Preeclampsia

Magnetic resonance imaging is a powerful tool in the diagnosis of missed or occult fractures on radiographic and computed tomographic (CT) imaging, through the detection of bone marrow edema. Although radiologists often rely on bone marrow edema as a guide for diagnosing subtle underlying fractures, it is important to recognize its limitations as a diagnostic metric. We present a rare case demonstrating the absence of bone marrow edema after acute trauma and confirmed Lisfranc fracture in a patient with preeclampsia and propose an interesting physiologic mechanism to explain this manifestation.

Radiologists and Clinical Trials: Part 1 The Truth About Reader Disagreements

The debate over human visual perception and how medical images should be interpreted have persisted since X-rays were the only imaging technique available. Concerns over rates of disagreement between expert image readers are associated with much of the clinical research and at times driven by the belief that any image endpoint variability is problematic. The deeper understanding of the reasons, value, and risk of disagreement are somewhat siloed, leading, at times, to costly and risky approaches, especially in clinical trials. Although artificial intelligence promises some relief from mistakes, its routine application for assessing tumors within cancer trials is still an aspiration. Our consortium of international experts in medical imaging for drug development research, the Pharma Imaging Network for Therapeutics and Diagnostics (PINTAD), tapped the collective knowledge of its members to ground expectations, summarize common reasons for reader discordance, identify what factors can be controlled and which actions are likely to be effective in reducing discordance. Reinforced by an exhaustive literature review, our work defines the forces that shape reader variability. This review article aims to produce a singular authoritative resource outlining reader performance's practical realities within cancer trials, whether they occur within a clinical or an independent central review.