Interpretation of abdominal CT: analysis of errors and their causes

Abdominopelvic CT in COVID-19 patients with abdominal complaints including both waves and controls: reader agreement and overcalls after consensus review

BACKGROUND

Since many COVID-19 publications lack consensus reviews or controls, interpretive accuracy is unclear; abdominal processes unique or infrequent during the pandemic remain unknown. The incidence and nature of CT findings accounting for abdominal complaints in COVID patients, reader agreement and overcalling will be determined.

METHODS

A retrospective study was performed on COVID patients with abdominal complaints from 3/15/2020-5/31/2020 and 11/1/2020-4/15/2021 including matched controls. Reviewers blinded to initial reads interpreted abdominopelvic CT exams, with discordant cases resolved in consensus. Reader agreement was measured by Cohen's Kappa, differences between cohorts by permutation tests and factors affecting false positive/negative rates by Fisher's Exact Test and logistic regression.

RESULTS

116 first wave (average age 65 years [±15.3], 63 [54%] women) and 194 second wave COVID cases (average age 64 years [±16.3], 103 [53%] women) including 116 wave 1 and 194 wave 2 prepandemic controls were included. Concordance was lower among COVID cases than controls (Cohen's Kappa of 0.58 vs. 0.82 [p ≤ 0.001]) and among wave 1 than wave 2 cases (Cohen's Kappa of 0.45 vs. 0.66 [p = 0.052]). With true positives defined as consensus between the initial reader and study reader, false positive rates were higher among COVID cases than controls (OR = 0.42, p = 0.003) and for initial than study reader (OR = 0.36, p ≤ 0.001), but lower in wave 2 than 1 (OR = 0.5, p = 0.028).

CONCLUSION

Greater reader disagreement occurred during COVID than prepandemic with no reader bias as both initial and study readers called more false positives among COVID cases than controls. More overcalling occurred during COVID with colitis and cystitis most common.

Discrepancy rate and clinical impact of preliminary reports from radiology residents

Background

Residents usually cover night and weekend shifts issuing the preliminary reading of radiological studies in university hospitals. This is essential to strengthening decision-making skills when facing complex cases independently. However, there should be a balance between patient safety and academic experience since some concern has been expressed about the accuracy of the interpretations generated by trainees. This work aims to evaluate and characterize the discrepancies in preliminary reports issued by radiology residents.

Material and methods

Radiologists filled out a questionnaire to evaluate preliminary reports of trainees considering diagnosis, findings description, clinical approach changes, and critical findings. Analysis was performed considering modality, imaging type, body part, and resident academic year. A Chi-square test with a significance level α of 0.05 was used to make group comparisons.

Results

A total of 9072 studies were reviewed. Major and minor overall discrepancy rates were 1.7% and 8.3%, respectively. Minor discrepancy rate, findings description, and critical findings identification improved with increasing academic year, both overall and by modality. Discrepancy rates were lower for CT than MR and neuroimaging than for body-imaging studies. The highest major and minor discrepancy rates as abdomen/pelvis CT and lumbar-spine MR, respectively. Two percent of reports presented discrepancies that could generate a medical approach change.

Conclusion

Discrepancy rates are low and comparable with those reported in the literature. These rates tend to improve as the resident's academic year increases. Our results suggest that radiology residents' coverage of night shifts and weekends is a practice that benefits the educational process without negatively impacting patient safety.

Impact of Shift Volume on Neuroradiology Diagnostic Errors at a Large Tertiary Academic Center

BACKGROUND AND PURPOSE

Medical errors can result in significant morbidity and mortality. The goal of our study is to evaluate correlation between shift volume and errors made by attending neuroradiologists at an academic medical center, using a large data set.

MATERIALS AND METHODS

CT and MRI reports from our Neuroradiology Quality Assurance database (years 2014 - 2020) were searched for attending physician errors. Data were collected on shift volume, category of missed findings, error type, interpretation setting, exam type, clinical significance.

RESULTS

654 reports contained diagnostic error. There was a significant difference between mean volume of interpreted studies on shifts when an error was made compared with shifts in which no error was documented (46.58 (SD=22.37) vs 34.09 (SD=18.60), p<0.00001); and between shifts when perceptual error was made compared with shifts when interpretive errors were made (49.50 (SD=21.9) vs 43.26 (SD=21.75), p=0.0094). 59.6% of errors occurred in the emergency/inpatient setting, 84% were perceptual and 91.1% clinically significant. Categorical distribution of errors was: vascular 25.8%, brain 23.4%, skull base 13.8%, spine 12.4%, head/neck 11.3%, fractures 10.2%, other 3.1%. Errors were detected most often on brain MRI (25.4%), head CT (18.7%), head/neck CTA (13.8%), spine MRI (13.7%).

CONCLUSION

Errors were associated with higher volume shifts, were primarily perceptual and clinically significant. We need National guidelines establishing a range of what is a safe number of interpreted cross-sectional studies per day.

Individualized and generalized models for predicting observer performance on liver metastasis detection using CT

Purpose: Radiologists exhibit wide inter-reader variability in diagnostic performance. This work aimed to compare different feature sets to predict if a radiologist could detect a specific liver metastasis in contrast-enhanced computed tomography (CT) images and to evaluate possible improvements in individualizing models to specific radiologists. Approach: Abdominal CT images from 102 patients, including 124 liver metastases in 51 patients were reconstructed at five different kernels/doses using projection domain noise insertion to yield 510 image sets. Ten abdominal radiologists marked suspected metastases in all image sets. Potentially salient features predicting metastasis detection were identified in three ways: (i) logistic regression based on human annotations (semantic), (ii) random forests based on radiologic features (radiomic), and (iii) inductive derivation using convolutional neural networks (CNN). For all three approaches, generalized models were trained using metastases that were detected by at least two radiologists. Conversely, individualized models were trained using each radiologist's markings to predict reader-specific metastases detection. Results: In fivefold cross-validation, both individualized and generalized CNN models achieved higher area under the receiver operating characteristic curves (AUCs) than semantic and radiomic models in predicting reader-specific metastases detection ability ( p < 0.001 ). The individualized CNN with an AUC of mean (SD) 0.85(0.04) outperformed the generalized one [ AUC = 0.78 ( 0.06 ) , p = 0.004 ]. The individualized semantic [ AUC = 0.70 ( 0.05 ) ] and radiomic models [ AUC = 0.68 ( 0.06 ) ] outperformed the respective generalized versions [semantic AUC = 0.66 ( 0.03 ) , p = 0.009 ; radiomic AUC = 0.64 ( 0.06 ) , p = 0.03 ]. Conclusions: Individualized models slightly outperformed generalized models for all three feature sets. Inductive CNNs were better at predicting metastases detection than semantic or radiomic features. Generalized models have implementation advantages when individualized data are unavailable.

Mandating Limits on Workload, Duty, and Speed in Radiology

Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.

The effects of examiner fatigue on the diagnostic accuracy of dental radiographs

OBJECTIVES

The aim of the study was to find out whether and to what extent the performance of dentists regarding diagnostic evaluation of dental radiographs is influenced by symptoms of fatigue.

MATERIALS AND METHODS

Over a period of 40 minutes, 21 dentists evaluated a database of 96 randomly selected, infinitely repeating intraoral dental radiographs for the presence of periapical radiolucencies. Both before and after, participants were asked to assess their subjective fatigue using the Swedish Occupational Fatigue Inventory (SOFI), Visual Analogue Scale (VAS), and Numerical Rating Scale (NRS). Diagnostic accuracy was analyzed using the Receiver Operating Characteristics (ROC) method. Furthermore, the correlation between diagnostic accuracy and radiographic experience, image viewing time, and level of training was also evaluated.

RESULTS

The study showed that despite increasing fatigue, the diagnostic accuracy of the examiners remained consistent with an average AUC value of 0.768 ± 0.091. Within the 40-min reporting period, no statistically significant fluctuations were found. The diagnostic accuracy varied depending on the radiographic experience: with many years of radiographic experience, the diagnostic accuracy increased. At the same time, the older study participants with greater radiographic experience became less tired compared to younger study participants during the examination.

CONCLUSIONS

Although an increase in fatigue was observed during the 40-min examination, the diagnostic accuracy of the doctors remained constant.

CLINICAL RELEVANCE

Due to the high workload which needs to be handled in a limited time, medical and dental professionals have reached a certain level of exposition to stress that can lead to physical fatigue. However, the increasing fatigue should not negatively influence the work of the doctors. The study shows that the radiodiagnostic accuracy remained the same.

The Impact of Fatigue on Complex CT Case Interpretation by Radiology Residents

RATIONALE AND OBJECTIVES

To examine the effect of fatigue on diagnostic performance and interpretation of complex CT trauma studies by radiology residents.

MATERIALS AND METHODS

Ten 2nd, 3rd, and 4th year radiology residents were given 1 hour to review as many complex CT cases as they could, once when they were fresh and once when they were fatigued (after a full day or night of diagnostic reading activity). Number of cases completed, interpretation accuracy and time per case were analyzed.

RESULTS

On average readers completed 7.5 cases when fresh and 7.3 when fatigued, with 4th and 3rd year residents spending more time per case when fresh than fatigued and 2nd year residents spending more time when fatigued. The total number of true positives for major (fresh mean = 29.4 vs fatigued mean = 24.1) and minor (fresh mean = 23.30 vs fatigued mean = 18.90) findings decreased for fresh vs fatigued; and there was a significant difference as a function of year of residency (F = 4.72, p = 0.027). The 4th year residents had the most TPS in both conditions, followed by 3rd then 2nd year residents. There were more false positives when fatigued than fresh.

CONCLUSION

Fatigue significantly impacts radiologic diagnostic efficiency and efficacy, and differs as a function of year of residency.

Internet of Medical Things: An Effective and Fully Automatic IoT Approach Using Deep Learning and Fine-Tuning to Lung CT Segmentation

Several pathologies have a direct impact on society, causing public health problems. Pulmonary diseases such as Chronic obstructive pulmonary disease (COPD) are already the third leading cause of death in the world, leaving tuberculosis at ninth with 1.7 million deaths and over 10.4 million new occurrences. The detection of lung regions in images is a classic medical challenge. Studies show that computational methods contribute significantly to the medical diagnosis of lung pathologies by Computerized Tomography (CT), as well as through Internet of Things (IoT) methods based in the context on the health of things. The present work proposes a new model based on IoT for classification and segmentation of pulmonary CT images, applying the transfer learning technique in deep learning methods combined with Parzen’s probability density. The proposed model uses an Application Programming Interface (API) based on the Internet of Medical Things to classify lung images. The approach was very effective, with results above 98% accuracy for classification in pulmonary images. Then the model proceeds to the lung segmentation stage using the Mask R-CNN network to create a pulmonary map and use fine-tuning to find the pulmonary borders on the CT image. The experiment was a success, the proposed method performed better than other works in the literature, reaching high segmentation metrics values such as accuracy of 98.34%. Besides reaching 5.43 s in segmentation time and overcoming other transfer learning models, our methodology stands out among the others because it is fully automatic. The proposed approach has simplified the segmentation process using transfer learning. It has introduced a faster and more effective method for better-performing lung segmentation, making our model fully automatic and robust.

The Back Alleys and Dark Corners of Abdomen and Pelvis Computed Tomography: The Most Frequent Sites of Missed Findings in the Multiplanar Era

Objectives

Radiologists reading multiplanar abdominal/pelvic computed tomography (CT) are vulnerable to oversight of specific anatomic areas, leading to perceptual errors (misses). The aims of this study are to identify common sites of major perceptual error at our institution and then to put these in context with earlier studies to produce a comprehensive overview.

Material and Methods

We reviewed our quality assurance database over an 8-year period for cases of major perceptual error on CT examinations of the abdomen and pelvis. A major perceptual error was defined as a missed finding that had altered management in a way potentially detrimental to the patient. Record was made of patient age, gender, study indication, study priority (stat/routine), and use of IV and/or oral contrast. Anatomic locations were subdivided as lung bases, liver, pancreas, kidneys, spleen, mesentery, peritoneum, retroperitoneum, small bowel, colon, appendix, vasculature, body wall, and bones.

Results

A total of 216 missed findings were identified in 201 patients. The most common indication for the study was cancer follow-up (71%) followed by infection (11%) and abdominal pain (6%). The most common anatomic regions of error were the liver (15%), peritoneum (10%), body wall (9%), retroperitoneum (8%), and mesentery (6%). Data from other studies were reorganized into congruent categories for comparison.

Conclusion

This study demonstrates that the most common sites of significant missed findings on multiplanar abdominal/pelvic CT included the mesentery, peritoneum, body wall, bowel, vasculature, and the liver in the arterial phase. Data from other similar studies were reorganized into congruent categories to provide a comprehensive overview.

Integration of Peer Review in PACS Results in a Marked Increase in the Discrepancies Reported

OBJECTIVE. The objective of this article is to assess the impact of integrating peer review in PACS on the reporting of discrepancies. Our hypothesis is that a PACS-integrated machine-randomized and semiblinded peer review tool leads to an increase in discrepancies reported. MATERIALS AND METHODS. A PACS tool was implemented to prompt radiologists to perform peer review of prior comparison studies in a randomized fashion. The reviewed radiologist's name was omitted from the prior report in PACS. Before this implementation, radiologists entered peer reviews directly on the RADPEER website. Three academic subspecialty sections comprising 24 radiologists adopted the tool (adopters group). Three sections comprising 14 radiologists did not adopt the tool (nonadopters group). Peer review submissions were analyzed for 4 months before and 4 months after the implementation. The mean rate of significant discrepancies (RADPEER score 2b or higher) reported per radiologist was calculated and the discrepancy rates of the periods before and after the implementation were compared. RESULTS. The mean significant discrepancy rate reported per radiologist in the adopters group increased from 0.19% ± 0.46% (SD) before the implementation to 0.93% ± 1.45% after implementation (p = 0.01). No significant discrepancies were reported by the nonadopters group in either period. CONCLUSION. In this single institutional retrospective analysis, integrating peer review in PACS resulted in a fivefold increase in reported significant discrepancies. These results suggest that peer review data are influenced by the design of the tool used including PACS integration, randomization, and blinding.

Added value of double reading in diagnostic radiology,a systematic review

Objectives

Double reading in diagnostic radiology can find discrepancies in the original report, but a systematic program of double reading is resource consuming. There are conflicting opinions on the value of double reading. The purpose of the current study was to perform a systematic review on the value of double reading.

Methods

A systematic review was performed to find studies calculating the rate of misses and overcalls with the aim of establishing the added value of double reading by human observers.

Results

The literature search resulted in 1610 hits. After abstract and full-text reading, 46 articles were selected for analysis. The rate of discrepancy varied from 0.4 to 22% depending on study setting. Double reading by a sub-specialist, in general, led to high rates of changed reports.

Conclusions

The systematic review found rather low discrepancy rates. The benefit of double reading must be balanced by the considerable number of working hours a systematic double-reading scheme requires. A more profitable scheme might be to use systematic double reading for selected, high-risk examination types. A second conclusion is that there seems to be a value of sub-specialisation for increased report quality. A consequent implementation of this would have far-reaching organisational effects.

Key Points

• In double reading, two or more radiologists read the same images.

• A systematic literature review was performed.

• The discrepancy rates varied from 0.4 to 22% in various studies.

• Double reading by sub-specialists found high discrepancy rates.

Electronic supplementary material

The online version of this article (10.1007/s13244-018-0599-0) contains supplementary material, which is available to authorised users.

Trauma whole-body MDCT: an assessment of image quality in conventional dual-phase and modified biphasic injection

OBJECTIVE

To compare the image quality of conventional arterial and portal venous (PV) phase multidetector CT (MDCT) with two biphasic injection protocols in polytrauma patients.

METHODS

60 consecutive patients with polytrauma underwent body 256-slice MDCT with a conventional protocol or 1 of 2 single-pass biphasic protocols: Group A, arterial (30 s) and PV (60 s) phase acquisitions; Group B, "biphasic" contrast injection with a single acquisition at 60 s; and Group C, "modified biphasic" injection with a single acquisition at a 70-s delay. Images were analyzed for arterial, venous and parenchymal attenuation profiles with regions of interest in the major arteries, veins and solid abdominal organs. A 5-point scoring system was used to assess the image quality, with 5 representing excellent arterial, venous and parenchymal opacification and <3 representing non-diagnostic opacification. In addition, the effective dose (millisieverts) was compared between the groups.

RESULTS

In 93% of patients, image quality was scored as good or excellent (≥4). All studies were of satisfactory diagnostic quality. Overall, venous and arterial attenuation profiles were comparable. Attenuation profiles in the solid abdominal viscera were significantly higher (p < 0.01) using both biphasic protocols than with arterial or PV phase of conventional protocols. Effective doses were higher in Group A.

CONCLUSION

Comparable image quality can be achieved using a biphasic i.v. contrast injection protocol with single MDCT acquisition with less radiation and reduction in acquisition time.

ADVANCES IN KNOWLEDGE

For these particular biphasic injection protocols, we have shown that image quality is comparable with a conventional protocol. This has been achieved by comparing enhanced densities of specific structures, as well as gestalt scoring by assessors, on a 256-slice MDCT.

Variability in Observer Performance Between Faculty Members and Residents Using Breast Imaging Reporting and Data System (BI-RADS)-Ultrasound, Fifth Edition (2013)

Background

Ultrasonography (US) is a useful tool for breast imaging, yet is highly operator-dependent.

Objectives

We evaluated inter-observer variability and performance discrepancies between faculty members and radiology residents when describing breast lesions, by the fifth edition of breast imaging reporting and data system (BI-RADS)-US lexicon, and then attempted to identify whether inter-observer variability could be improved after one education session.

Patients and Methods

In total, 50 malignant lesions and 70 benign lesions were considered in our retrospective study. Two faculty members, two senior residents, and two junior residents separately assessed the US images. After the first assessment, the readers received one education session, and then reassessed the images in a random order. Inter-observer variability was measured using the kappa coefficient (κ). Performance discrepancy was evaluated by receiver operating characteristic (ROC) curves.

Results

For the faculty members, fair-to-good agreement was obtained in all descriptors and final assessment, while for residents, poor-to-moderate agreement was obtained. The areas under the ROC curves were 0.78 for the faculty members, 0.59 for the senior residents, and 0.52 for the junior residents, respectively. Diagnostic performance was significantly higher in the faculty members than the senior and junior residents (P = 0.0001 and < 0.0001, respectively). After one education session, the agreement in the final assessment was one level higher in the faculty members and senior residents, yet in the senior residents, the degree of agreement was still only fair. Moreover, in the junior residents, there was no improvement.

Conclusion

Investigative assessment of breast US by residents is inadvisable. We recommend continued professional resident training to improve the degree of agreement and performance.

Improving Patient Care Through Medical Image Perception Research

The interpretation of medical images across medical specialties is critical to patient care. As technology changes, so does health care, and clinicians today are increasingly viewing medical images in a variety of environments. Although access to such data is useful, even clinicians with expertise in image interpretation make errors. These errors may become more frequent as clinician workdays become longer and the number of images to be interpreted becomes larger. To prevent errors in medical image interpretation, we need to understand the underlying perceptual and cognitive mechanisms that guide image interpretation. We can then use what is learned to develop better training methods, automated image analysis, and processing tools. We can devise methods to reduce clinician fatigue and stress, and develop practice guidelines thereby improving patient care and outcomes.

The clinical impact of subspecialized radiologist reinterpretation of abdominal imaging studies, with analysis of the types and relative frequency of interpretation discrepancies

PURPOSE

The primary objective of this study was to determine the clinical impact and value of abdominal imaging reinterpretations by subspecialized abdominal imagers.

METHODS

Secondary interpretations for computed tomography (CT), magnetic resonance (MR), and ultrasound (US) abdominal studies performed outside our institution over a 7-month period were retrospectively compared to the primary (outside) interpretation, with interpretive differences recorded. Clinical notes, pathology and subsequent imaging determined ground truth diagnosis and the clinical impact of any interpretive discrepancies were graded as having high, medium, or little/no clinical impact. Interpretive comparisons were scored into categories: (1) no difference; (2) incidental findings of no clinical impact; (3) finding not reported; (4) significance of finding undercalled; (5) significance of finding overcalled; (6) finding misinterpreted; and (7) multiple discrepancy types in one report.

RESULTS

398 report comparisons were reviewed on 380 patients. There were 300 CT, 60 MR, and 38 US examinations. The primary report had 5.0% (20/398) high clinical impact interpretive discrepancies and 7.5% (30/398) medium clinical impact discrepancies. The subspecialized secondary report had no high clinical impact discrepancies and 8/398 (2.0%) medium clinical impact discrepancies. In order of frequency, high and medium impact discrepancies in the primary report consisted of 50% overcalls, 26% unreported findings, 18% undercalls, 4% misinterpretations, and 2% multiple discrepancies.

CONCLUSIONS

Subspecialty review of abdominal imaging exams can provide clinical benefit. Half of the discrepancies in this series of abdominal reinterpretations were due to overcalls.

The discrepancy rate between preliminary and official reports of emergency radiology studies: a performance indicator and quality improvement method

BACKGROUND

At teaching hospitals, radiology residents give preliminary reports for imaging studies requested from the Emergency Department (ED). Discrepancy rates between preliminary and final reports represent an important performance indicator.

PURPOSE

To present a system for feedback and follow-up of discrepancies, identify the variables associated with the rate and severity of such discrepancies, target the weaknesses, and suggest the need of a standard reference value for comparison among institutions.

MATERIAL AND METHODS

A monitoring and communication system between the Department of Diagnostic Radiology and Emergency Department was initiated to mark and follow all studies from the ED for which the official reading was different than the preliminary interpretation. Data analysis was performed on all studies from 1 June 2011 to 31 May 2012, based on the severity of the discrepancy, imaging modality, resident training level, and organ system. The distribution of the number of discrepancies among the different resident levels and imaging modalities was determined, as well as the distribution of three severity scores in correlation with other variables.

RESULTS

The overall discrepancy rate was 1.62%. The discrepancy rate was higher for first and second year residents (1.62% and 1.96%) than for third and fourth year residents (1.35% and 1.24%). It was higher for computed tomography (2.13%) than for radiographs (1.29%) and ultrasound (0.8%) (P value < 0.01), and higher for musculoskeletal (1.61%) than non-musculoskeletal (0.99%) radiographs (P value = 0.0003). Discrepancies with severity score one constituted 35.5% of the total discrepancies, those with severity scores two and three constituted 22.9% and 41.6%, respectively.

CONCLUSION

We have demonstrated a system for follow-up of discrepancy in interpreting emergency radiology studies, and recorded the discrepancy rate, with further analysis based on different variables. In terms of quality assurance, a periodical analysis might help to reduce the number of discrepant reports by targeted intervention.

Residents' performance in the interpretation of on-call "triple-rule-out" CT studies in patients with acute chest pain

RATIONALE AND OBJECTIVES

To evaluate the performance of radiology residents in the interpretation of on-call, emergency "triple-rule-out" (TRO) computed tomographic (CT) studies in patients with acute chest pain.

MATERIALS AND METHODS

The study was institutional review board-approved and Health Insurance Portability and Accountability Act compliant. Data from 617 on-call TRO studies were analyzed. Dedicated software enables subspecialty attendings to grade discrepancies in interpretation between preliminary trainee reports and their final interpretation as "unlikely to be significant" (minor discrepancies) or "likely to be significant" for patient management (major discrepancies). The frequency of minor, major and all discrepancies in resident's TRO interpretations was compared to 609 emergent non-electrocardiography (ECG)-synchronized chest CT studies using Pearson χ(2) test.

RESULTS

Minor discrepancies occurred more often in the TRO group (9.1% vs. 3.9%, P < .001), but there was no difference in the frequency of major discrepancies (2.1% vs. 2.8%, P = .55). Minor discrepancies in the TRO group most commonly resulted from missed extrathoracic findings with missed liver lesions being the most frequent. Major discrepancies mostly encompassed cardiac and extracardiac vascular findings but did not result in unnecessary interventions, significant immediate changes in management, or adverse patient outcomes.

CONCLUSIONS

On-call resident interpretation of TRO CT studies in patients with acute chest pain is congruent with final subspecialty attending interpretation in the overwhelming majority of cases. The rate of discrepancies likely to affect patient management in this domain is not different from emergent non-ECG-synchronized chest CT.

CT in adults: systematic review and meta-analysis of interpretation discrepancy rates

PURPOSE

To use meta-analysis to determine the discrepancy rate when interpreting computed tomography (CT) studies performed in adult patients and to determine whether discrepancy rate differs on the basis of body region or level of radiologist training.

MATERIALS AND METHODS

MEDLINE and EMBASE were searched from 1946 to June 2012 by using the combination "radiology AND (error OR peer review)." Two reviewers independently selected studies that met the inclusion criteria and extracted study data. Total and major discrepancy rates were investigated with a random-effects meta-analysis, and subgroups were compared by using the χ(2) Q statistic. Subgroup analyses were performed on the basis of the level of training of the initial radiologist and the body system scanned.

RESULTS

Fifty-eight studies met the inclusion criteria (388 123 CT examinations). The pooled total discrepancy rate was 7.7% (95% confidence interval [CI]: 5.6%, 10.3%), and the major discrepancy rate was 2.4% (95% CI: 1.7%, 3.2%). The pooled major discrepancy rate was comparable for staff (2.9%; 95% CI: 1.2%, 6.7%) and residents (2.2%; 95% CI: 1.7%, 2.9%) (Q = 0.92, P = .633). The pooled major discrepancy rates for head CT (0.8%; 95% CI: 0.4%, 1.6%) and spine CT (0.7%; 95% CI: 0.2%, 2.7%) were lower than those for chest CT (2.8%; 95% CI: 1.5%, 5.4%) and abdominal CT (2.6%; 95% CI: 1.0%, 6.7%) (Q = 8.28, P = .041). Lack of blinding of the reference radiologist to the initial report was associated with a lower major discrepancy rate (2.0%; 95% CI: 1.4%, 2.7%; 43 studies) than when blinding was present (12.1%; 95% CI: 4.4%, 29.4%; five studies) (Q = 10.65, P = .001).

CONCLUSION

Potentially useful reference ranges were identified in the subgroup analyses on the basis of body region scanned at adult CT. However, considerable heterogeneity that is only partially explained by subgroup analysis signifies that further research is necessary--particularly regarding the question of blinding of the reference radiologist.

Medical imaging displays and their use in image interpretation

The adequate and repeatable performance of the image display system is a key element of information technology platforms in a modern radiology department. However, despite the wide availability of high-end computing platforms and advanced color and gray-scale monitors, the quality and properties of the final displayed medical image may often be inadequate for diagnostic purposes if the displays are not configured and maintained properly. In this article-an expanded version of the Radiological Society of North America educational module "Image Display"-the authors discuss fundamentals of image display hardware, quality control and quality assurance processes for optimal image interpretation settings, and parameters of the viewing environment that influence reader performance. Radiologists, medical physicists, and other allied professionals should strive to understand the role of display technology and proper usage for a quality radiology practice. The display settings and display quality control and quality assurance processes described in this article can help ensure high standards of perceived image quality and image interpretation accuracy.

Evaluating the accuracy of emergency medicine resident interpretations of abdominal CTs in patients with non-traumatic abdominal pain

Abdominal computed tomography (CT) is widely used as a diagnostic tool in emergency medicine (EM) to accurately diagnose abdominal pain. EM residents must be able to offer preliminary interpretations of CT imaging. In this study, we evaluated the preliminary interpretation ability of a sample of emergency residents presented with adult abdominal CT images, and compared their results with those of radiology residents. We conducted a prospective observational study from November 16, 2008 to June 30, 2009. During this time, we gathered preliminary interpretations of consecutive abdominal CT made by emergency and radiology residents. We assessed the discrepancy rates of both samples by comparing their findings to the final reports from attending radiologists. A total of 884 cases were enrolled in the present study. The discrepancy rates of emergency and radiology residents were 16.7% and 12.2%, respectively. When female genital organs, peritoneum, adrenal glands, or the musculoskeletal system were abnormal, we found that emergency residents' preliminary interpretations of CT images were insufficient compared to those of radiology residents. Therefore more formal education is needed to emergency residents. If possible, the preliminary interpretations of radiology attending physicians are ideal until improving the ability of interpretations of emergency residents in abdomen CT.

Blind spots at oncological CT: lessons learned from PET/CT

Improved accuracy in oncological computed tomography (CT) could lead to a decrease in morbidity and improved survival for oncology patients. Visualization of metabolic activity using the glucose analogue [¹⁸F]fluorodeoxyglucose (FDG) in combination with the high anatomic resolution of CT in an integrated positron emission tomography (PET)/CT examination has the highest sensitivity and specificity for the detection of primary and metastatic lesions. However, PET/CT costs are high and patient access is limited; thus CT remains the primary imaging modality in oncology patients. We have noted that subtle lesions are more easily detected on CT by radiologists with PET/CT experience. We aimed to provide a brief review of the literature with comparisons of multi-detector computed tomography (MDCT) and PET/CT in primary and metastatic disease with an emphasis on findings that may be overlooked on MDCT in cancer of the breast, lung, colon, and ovaries, and in melanoma, as well as thrombosis in oncology patients. We further reviewed our experience for illustrative comparisons of PET/CT and MDCT studies. Experience in interpreting conventional CT scans alongside PET/CT can help the reader develop an appreciation for the subtle appearance of some lesions on CT that might otherwise be missed. This could improve detection rates, reduce errors, and improve patient management.

Uncommon causes of acute abdominal pain: multidetector computed tomography pearls and pitfalls for the radiologist on call

We review uncommon causes of acute abdominal pain in which inconclusive multidetector computed tomography (MDCT) studies were followed by emergency surgery and unexpected diagnoses. Despite dedicated protocols and technical advances, MDCT of uncommon causes of acute abdominal pain still represents a significant challenge for the radiologist on call. We emphasize diagnostic pearls and pitfalls that may help the radiologist on call identify or suspect these uncommon causes of acute abdominal pain on MDCT.

Do long radiology workdays affect nodule detection in dynamic CT interpretation?

PURPOSE

A previous study demonstrated decreased diagnostic accuracy for finding fractures and decreased ability to focus on skeletal radiographs after a long working day. Skeletal radiographic examinations commonly have images that are displayed statically. The aim of this study was to investigate whether diagnostic accuracy for detecting pulmonary nodules on CT of the chest displayed dynamically would be similarly affected by fatigue.

METHODS

Twenty-two radiologists and 22 residents were given 2 tests searching CT chest sequences for a solitary pulmonary nodule before and after a day of clinical reading. To measure search time, 10 lung CT sequences, each containing 20 consecutive sections and a single nodule, were inspected using free search and navigation. To measure diagnostic accuracy, 100 CT sequences, each with 20 sections and half with nodules, were displayed at preset scrolling speed and duration. Accuracy was measured using receiver operating characteristic curve analysis. Visual strain was measured via dark vergence, an indicator of the ability to keep the eyes focused on the display.

RESULTS

Diagnostic accuracy was reduced after a day of clinical reading (P = .0246), but search time was not affected (P > .05). After a day of reading, dark vergence was significantly larger and more variable (P = .0098), reflecting higher levels of visual strain, and subjective ratings of fatigue were also higher.

CONCLUSIONS

After their usual workday, radiologists experience increased fatigue and decreased diagnostic accuracy for detecting pulmonary nodules on CT. Effects of fatigue may be mitigated by active interaction with the display.

The use of pre-operative computed tomography in the assessment of the acute abdomen

INTRODUCTION

While there are a lot of data on the accuracy of computed tomography (CT) in diagnosing specific causes of an acute abdomen, there is very little information on the accuracy of CT in the acute general surgical admissions workload. We look at the diagnostic accuracy of CT in patients presenting with an acute abdomen who ultimately required a laparotomy.

METHODS

Patients who underwent an emergency laparotomy between 2008 and 2010 at Eastbourne District General Hospital with pre-operative CT on the same admission were included in the study. The CT report was compared with the laparotomy and histology findings and, where a discrepancy existed, the original imaging was reviewed by a senior consultant blinded to the original report and laparotomy findings.

RESULTS

A total of 196 emergency laparotomies were performed over the 2-year period, with 112 patients undergoing preoperative CT. Fifteen patients were excluded from the study due to missing notes. In the remaining 97 patients, 80 CT reports correlated with the final operative diagnosis, giving a diagnostic accuracy of 82%. Of these, the on-call registrar was the initial reporter in 37 scans, with a diagnostic accuracy of 78%. On review of the CT by a second consultant, this increased to 90 correlations, yielding an accuracy of 93%. Delay between CT and the operation did not significantly alter diagnostic accuracy, nor was there any statistically significant reduction in accuracy in reports issued by on-call registrars.

CONCLUSIONS

On first reporting, CT misses 18% of diagnoses that ultimately require operative intervention. Reducing the threshold for obtaining a second consultant radiologist review significantly improves the diagnostic accuracy to 93%.

Abdominal pain in the ED: a 35 year retrospective

OBJECTIVE

Research published in 1972 and 1993 has detailed the demographics, diagnoses, and diagnostic test utilization of adult patients presenting with nontraumatic abdominal pain to the emergency department (ED) at the University of Virginia Hospital. This is an update of those studies, designed to examine the present state of diagnosis and management of abdominal pain, as well as to look at trends during the 35-year span of the investigations.

METHODS

One thousand consecutive adult patients presenting in the year 2007 with abdominal pain as their chief complaint were included in the analysis. Demographic data, discharge diagnosis, disposition, ED length of stay, charges, and diagnostic test utilization information were gathered and analyzed using electronic databases.

RESULTS

These patients represented 6.5% of the total ED census. Sixty-five percent were female, 24.7% hospitalized, and 21% diagnosed with undifferentiated abdominal pain. Relative to 1993, there were more patients receiving specific diagnoses, (79% versus 75%) and a higher rate of hospitalization (24.7% versus 18.3%). Use of diagnostic testing has markedly increased in frequency, most notably computed tomography and ultrasound, which have risen 6-fold. One of these imaging modalities is now used in 42% of patient encounters. Visit times were longer and patient charges higher. There were 2 cases of missed surgical disease in 2007 compared with 1 in 1993 and 8 in 1972.

CONCLUSION

Over the past 35 years, ED management of atraumatic abdominal pain has become time, money, and resource intense. Widespread use of sophisticated imaging has had a small impact on diagnostic specificity but has not produced lower admission rates or fewer cases of missed surgical illness.

Overnight resident interpretation of torso CT at a level 1 trauma center an analysis and review of the literature

RATIONALE AND OBJECTIVES

At the authors' level 1 trauma center, postgraduate year 3 and 4 radiology residents interpret urgent overnight imaging studies, which are reviewed by attending radiologists the next morning. The goals of this study were to determine the discrepancy rate for torso computed tomography between resident radiologists' preliminary interpretations and attending radiologists' final interpretations and to identify adverse patient events secondary to the delayed diagnoses.

MATERIALS AND METHODS

All torso computed tomographic studies interpreted by weekday night residents (8 pm to 8 am) from January 1, 2005, to March 13, 2007, were evaluated retrospectively. Major discrepancies between the residents' interpretations and the attending radiologists' interpretations were compiled. Exams with major discrepancies were additionally reviewed by two radiology residents and an attending emergency radiologist. The medical records of patients with major discrepant findings were also reviewed to identify adverse events that occurred because of the delays in final interpretation.

RESULTS

A total of 4768 chest, abdominal, and/or pelvic computed tomographic studies were interpreted during the study period. A total of 112 major discrepancies were collected. In 17 cases (15%), two additional radiology residents and an attending emergency radiologist agreed with the initial residents' interpretations, decreasing the major discrepancy rate to 95 of 4768 (2.0%), consistent with data from the literature (0.4%-10%). Management was changed in 16 patients (0.3%) because of the major discrepancies: 13 patients underwent additional investigations, and 3 patients were recalled to the emergency room. No mortality or morbidity was directly attributed to the delays in diagnosis.

CONCLUSION

At the authors' institution, there was a 2.0% discrepancy rate between residents' preliminary interpretations and attending radiologists' final interpretations of overnight torso computed tomography, with a small rate of additional intervention as a result of the major discrepancies. No adverse patient effects were directly attributable to discrepant interpretations. In discrepant cases, there was a 15% disagreement rate (17 of 112) between attending radiologists, suggesting some degree of interreader variance in interpretation. In addition, this disagreement rate among attending radiologists may imply that these cases were complex. Because there is educational value in overnight call, independent radiology resident coverage should continue.

Clinical governance improves the quality of nuclear medicine reporting

OBJECTIVE

To assess the quality of nuclear medicine reporting, within a private UK hospital, of five physicians from four different National Health Service trusts and compare it with a similar previous clinical governance exercise.

METHODS

Reports (n=140) were shown anonymously to all five physicians, including the one who produced the report. Each physician ranked them on a scale of 1-5, with 1 and 5 corresponding to complete disagreement and complete agreement, respectively. All reports with at least one score of <4 were subjected to consensus review by all five physicians and subsequently given a consensus score.

RESULTS

Six hundred and ninety-one audit opinions were present out of a possible 700 (98.7%). Forty-three reports were reviewed, of which 11 received a consensus score of <4 (7.9%). This is not significantly different from the proportion of nontrivial errors in our earlier study (10.2%). Only three reports were present, however, with a score of <3 (2.1%), significantly fewer (P<0.02) than the proportion of nontrivial errors in our earlier study. No scores of 1 were recorded. No reporter attracted significantly more scores of <4 compared with the overall proportion of such scores. A score given by an auditing physician which was 2 or more points different from the consensus score was defined as a suboptimal audit. Forty-four of 691 suboptimal audits (6.4%) were present, significantly fewer than the proportion of suboptimal audits in our earlier study (9.7%; P<0.03).

CONCLUSION

Studies such as these provide a useful framework for monitoring performance. This improved significantly in this study as compared with our previous audit.

Discordance rates between preliminary and final radiology reports on cross-sectional imaging studies at a level 1 trauma center

RATIONALE AND OBJECTIVES

The goal was to determine discordance rates between preliminary radiology reports provided by on-call radiology house staff and final reports from attending radiologists on cross-sectional imaging studies requested by emergency department staff after hours.

MATERIALS AND METHODS

A triplicate carbon copy reporting form was developed to provide permanent records of preliminary radiology reports and to facilitate communication of discrepant results to the emergency department. Data were collected over 21 weeks to determine the number of discordant readings. Patients' medical records were reviewed to show whether discrepancies were significant or insignificant and to assess their impact on subsequent management and patient outcome.

RESULTS

The emergency department requested 2830 cross-sectional imaging studies after hours and 2311 (82%) had a copy of the triplicate form stored in radiology archives. Discrepancies between the preliminary and final report were recorded in 47 (2.0%), with 37 (1.6%) considered significant: 14 patients needed no change, 13 needed a minor change, and 10 needed a major change in subsequent management. Ten (0.43%) of the discordant scans were considered insignificant. A random sample of 104 (20%) of the 519 scans without a paper triplicate form was examined. Seventy-one (68%) did have a scanned copy of the triplicate form in the electronic record, with a discrepancy recorded in 3 (4.2%), which was not statistically different from the main cohort (P = .18).

CONCLUSION

Our study suggests a high level of concordance between preliminary reports from on-call radiology house staff and final reports by attending subspecialty radiologists on cross-sectional imaging studies requested by the emergency department.

Missed lesions at abdominal oncologic CT: lessons learned from quality assurance

The evaluation of oncology patients represents a substantial volume of the workload in many radiology departments. Interpreting the results of oncologic examinations is often challenging and time-consuming because many abnormalities are identified in the same examination and must be compared with the findings in previous studies. However, errors in the interpretation of oncologic computed tomographic (CT) scans can have significant effects on patient care. These effects may range from withdrawal from a clinical trial or cessation of therapy to repeat CT examination because of a technically inadequate study, CT-guided biopsy of newly identified lesions, or initiation of therapy for previously unrecognized lesions. A root cause analysis of reported errors in the interpretation of abdominal and pelvic CT scans led to the identification of potential pitfalls that may be encountered when interpreting oncologic CT scans and factors that contribute to these errors. Awareness of the spectrum of factors that contribute to misinterpretation of CT scans in oncology patients may improve the performance of the individual radiologist and ultimately translate into improved patient care.

Preliminary Radiology Resident Interpretations Versus Final Attending Radiologist Interpretations and the Impact on Patient Care in a Community Hospital

OBJECTIVE

At academic institutions, overnight emergency radiology examinations are interpreted by the on-call radiology resident and are reviewed by an attending radiologist in the morning. The objective of our study was to determine the rate of discrepancies between the two interpretations and the possible effect, if any, on patient care.

MATERIALS AND METHODS

The preliminary reports for 11,908 emergency diagnostic imaging examinations interpreted after hours by residents over a 3-year period (January 2002-January 2005) were reviewed retrospectively for any discrepancy with the attending radiologist's final interpretation. A discrepancy was noted if verbal notification of the ordering physician was required. The medical charts of the cases for which there was a major discrepancy between the two interpretations were reviewed. The discrepancies were categorized as to the effect on patient morbidity. The resident discrepancy rates were also compared with RADPEER data from our institution.

RESULTS

The overall major discrepancy rate was 2.6%. This rate is comparable to RADPEER data, which found a misinterpretation rate of 2.1%. The technique most commonly involved in cases with discrepant interpretations was contrast-enhanced CT of the abdomen and pelvis, with the most common diagnosis related to acute appendicitis (total of 21 cases). The rate of discrepancy was highest for residents who were in their third year of training. The indications for these examinations varied; however, the effect on patient management was no significant effect in 92.8%, some negative effect in 6.9%, and significant negative effect in 0.3%.

CONCLUSION

The results of this investigation highlight the minimal discrepancy rate that occurs with overnight resident coverage. Thus, there is no detrimental effect on the quality of patient care from relying on preliminary interpretations made by radiology residents.

Prospective comparison of helical CT of the abdomen and pelvis without and with oral contrast in assessing acute abdominal pain in adult Emergency Department patients

PURPOSE

This prospective study compares the agreement of nonenhanced helical computed tomography (NECT) with oral contrast-enhanced computed tomography (CECT) in Emergency Department (ED) patients presenting with acute abdominal pain.

MATERIALS AND METHODS

One hundred eighteen patients presenting to the ED with acute abdominal pain undergoing CT were enrolled over a 13-month period using convenience sampling. Exclusion criteria included acute trauma, pregnancy, unstable patients, and patients suspected of having urinary calculi. Patients were scanned helically using 5-mm collimation before and approximately 90 min after oral contrast administration. Both exams were prospectively interpreted by different attending radiologists in a blinded fashion using an explicit data sheet specifying the presence or absence of 28 parameters relating to various common diagnoses.

RESULTS

The 118 patients had a mean age of 49 years, a male: female ratio of 7:13, and a median height, weight, and BMI of 166 cm, 80 kg, and 29, respectively. The most common indications for the study included appendicitis (32%) and diverticular disease (12%). Pain maximally localized to the right lower quadrant in 37% and the left lower quadrant in 21%. There were 21 patients that had significant disagreement of interpretations between NECT and CECT resulting in a simple agreement of 79% (95% CI: 70-87%). For specific radiologic parameters, agreement ranged from 77 to 100%. A post hoc agreement analysis was subsequently performed by two radiologists and only five paired scans were identified as discordant between the NECT and CECT. For only one of these patients did both radiologists agree that there was a definite discordant result between the two studies. A final unblinded consensus review demonstrated that much of the disagreement between the interpretations was related to interobserver variation.

CONCLUSION

There is 79% simple agreement between NECT and CECT in diagnosing various causes of acute abdominal pain in adult ED patients. Post hoc analysis indicates that a significant portion of the discordance was attributable to interobserver variability. This data suggests that NECT should be considered in adult ED patients presenting with acute abdominal pain.

Extracolonic abnormalities discovered incidentally at CT colonography in a male population

PURPOSE

To evaluate prospectively the prevalence of incidental extracolonic findings at computed tomographic (CT) colonography and to estimate the cost of their imaging work-up in male patients with high and those with average risk of colorectal cancer.

MATERIALS AND METHODS

This study was approved by the institutional review board, and informed consent was obtained from all patients. The study was compliant with requirements of the Health Insurance Portability and Accountability Act. CT colonography was performed in 500 men (mean age, 62.5 years). Of these patients, 194 (38.8%) were at average risk for colorectal cancer and presented for routine screening. The other 306 (61.2%) were at high risk for colorectal cancer. Extracolonic findings were recorded and categorized as either clinically important or clinically unimportant. Clinically important findings were defined as those that necessitated further diagnostic studies or medical or surgical follow-up. The cost of additional imaging required to further characterize important lesions was estimated. Chart review was performed (mean length of follow-up, 3.6 years) to determine whether any important findings were missed at CT colonography. The Fisher exact test was used to determine whether there was a difference between the percentages of average- and high-risk patients with extracolonic findings.

RESULTS

Of the 500 patients in the study, 315 (63.0%) had extracolonic findings, and 45 (9.0%) had clinically important extracolonic findings. Of the 596 extracolonic findings identified, 50 (8.4%) were thought to be clinically important. The mean additional cost to work up important findings was $28.12 per CT colonographic examination. There were no significant differences between average-risk and high-risk patients in the percentages of extracolonic findings (P = .25) or clinically important extracolonic findings (P = .11).

CONCLUSION

A substantial number of both average- and high-risk patients undergoing CT colonography will be found to have clinically important extracolonic findings. There was no increased morbidity or mortality associated with the additional evaluation of extracolonic findings. The cost of evaluating these lesions is low, given the potential for positive effects on patient care.

A prospective study assessing the efficacy of abdominal computed tomography scan without bowel preparation in diagnosing intestinal wall and luminal lesions in patients presenting to the emergency room with abdominal complaints

AIM: To evaluate the positive predictive value of abdominal non-prepared computed tomography (CT) for diagnosing intestinal lumen or wall lesions in patients presenting to the emergency room (ER) with abdominal complaints.

METHODS: For 1-year we prospectively evaluated all ER patients hospitalized after abdominal CT scan detected either intraluminal or intestinal wall lesions. These patients underwent colonoscopy serving as gold standard. Patients with prior abdominal pathology or CT findings of appendicitis or diverticulitis were excluded.

RESULTS: Five hundred and sixty-eight abdominopelvic CT scans were performed in the ER, 96 had positive colonic findings. Sixty-two patients were excluded, 46 because of diverticulitis or appendicitis, 16 because of prior abdominal pathology. Of the remaining 34 patients, 14 did not undergo colonoscopy during hospitalization. Twenty eligible patients were included in the study. The positive predictive value of the CT scans performed in the ER was calculated to be 45% (95% CI 25-67).

CONCLUSION: CT findings correlated with colonoscopic findings only in approximately half of the cases. Relying on non-prepared CT scan findings in planning patient management and colonoscopy may lead to unnecessary diagnostic work-ups.