Effect of reconstruction methods and x-ray tube current-time product on nodule detection in an anthropomorphic thorax phantom: A crossed-modality JAFROC observer study

Purpose:

To evaluate nodule detection in an anthropomorphic chest phantom in computed tomography (CT) images reconstructed with adaptive iterative dose reduction 3D (AIDR^3D) and filtered back projection (FBP) over a range of tube current–time product (mAs).

Methods:

Two phantoms were used in this study: (i) an anthropomorphic chest phantom was loaded with spherical simulated nodules of 5, 8, 10, and 12 mm in diameter and +100, −630, and −800 Hounsfield units electron density; this would generate CT images for the observer study; (ii) a whole-body dosimetry verification phantom was used to ultimately estimate effective dose and risk according to the model of the BEIR VII committee. Both phantoms were scanned over a mAs range (10, 20, 30, and 40), while all other acquisition parameters remained constant. Images were reconstructed with both AIDR^3D and FBP. For the observer study, 34 normal cases (no nodules) and 34 abnormal cases (containing 1–3 nodules, mean 1.35 ± 0.54) were chosen. Eleven observers evaluated images from all mAs and reconstruction methods under the free-response paradigm. A crossed-modality jackknife alternative free-response operating characteristic (JAFROC) analysis method was developed for data analysis, averaging data over the two factors influencing nodule detection in this study: mAs and image reconstruction (AIDR^3D or FBP). A Bonferroni correction was applied and the threshold for declaring significance was set at 0.025 to maintain the overall probability of Type I error at α = 0.05. Contrast-to-noise (CNR) was also measured for all nodules and evaluated by a linear least squares analysis.

Results:

For random-reader fixed-case crossed-modality JAFROC analysis, there was no significant difference in nodule detection between AIDR^3D and FBP when data were averaged over mAs [F(1, 10) = 0.08, p = 0.789]. However, when data were averaged over reconstruction methods, a significant difference was seen between multiple pairs of mAs settings [F(3, 30) = 15.96, p < 0.001]. Measurements of effective dose and effective risk showed the expected linear dependence on mAs. Nodule CNR was statistically higher for simulated nodules on images reconstructed with AIDR^3D (p < 0.001).

Conclusions:

No significant difference in nodule detection performance was demonstrated between images reconstructed with FBP and AIDR^3D. mAs was found to influence nodule detection, though further work is required for dose optimization.

Mammography test sets: reading location and prior images do not affect group performance

AIM

To examine how the location where reading takes place and the availability of prior images can affect performance in breast test-set reading.

MATERIALS AND METHODS

Under optimized viewing conditions, 10 expert screen readers each interpreted a reader-specific set of images containing 200 mammographic cases. Readers, randomly divided into two groups read images under one of two pairs of conditions: clinical read with prior images and laboratory read with prior images; laboratory read with prior images and laboratory read without prior images. Region-of-interest (ROI) figure-of-merit (FOM) was analysed using JAFROC software. Breast side-specific sensitivity and specificity were tested using Wilcoxon matched-pairs signed rank tests. Agreement between pairs of readings was measured using Kendall's coefficient of concordance.

RESULTS

Group performances between test-set readings demonstrated similar ROI FOMs, sensitivity and specificity median values, and acceptable levels of agreement between pairs of readings were shown (W = 0.75-0.79, p < 0.001) for both pairs of reading conditions. On an individual reader level, two readers demonstrated significant decreases (p < 0.05) in ROI FOMs when prior images were unavailable. Reading location had an inconsistent impact on individual performance.

CONCLUSION

Reading location and availability of prior images did not significantly alter group performance.

Breast tomosynthesis and digital mammography: a comparison of diagnostic accuracy

OBJECTIVE

Our aim was to compare the ability of radiologists to detect breast cancers using one-view breast tomosynthesis (BT) and two-view digital mammography (DM) in an enriched population of diseased patients and benign and/or healthy patients.

METHODS

All participants gave informed consent. The BT and DM examinations were performed with about the same average glandular dose to the breast. The study population comprised patients with subtle signs of malignancy seen on DM and/or ultrasonography. Ground truth was established by pathology, needle biopsy and/or by 1-year follow-up by mammography, which retrospectively resulted in 89 diseased breasts (1 breast per patient) with 95 malignant lesions and 96 healthy or benign breasts. Two experienced radiologists, who were not participants in the study, determined the locations of the malignant lesions. Five radiologists, experienced in mammography, interpreted the cases independently in a free-response study. The data were analysed by the receiver operating characteristic (ROC) and jackknife alternative free-response ROC (JAFROC) methods, regarding both readers and cases as random effects.

RESULTS

The diagnostic accuracy of BT was significantly better than that of DM (JAFROC: p=0.0031, ROC: p=0.0415). The average sensitivity of BT was higher than that of DM (∼90% vs ∼79%; 95% confidence interval of difference: 0.036, 0.108) while the average false-positive fraction was not significantly different (95% confidence interval of difference: -0.117, 0.010).

CONCLUSION

The diagnostic accuracy of BT was superior to DM in an enriched population.

Quantifying the clinical relevance of a laboratory observer performance paradigm

OBJECTIVE

Laboratory observer performance measurements, receiver operating characteristic (ROC) and free-response ROC (FROC) differ from actual clinical interpretations in several respects, which could compromise their clinical relevance. The objective of this study was to develop a method for quantifying the clinical relevance of a laboratory paradigm and apply it to compare the ROC and FROC paradigms in a nodule detection task.

METHODS

The original prospective interpretations of 80 digital chest radiographs were classified by the truth panel as correct (C=1) or incorrect (C=0), depending on correlation with additional imaging, and the average of C was interpreted as the clinical figure of merit. FROC data were acquired for 21 radiologists and ROC data were inferred using the highest ratings. The areas under the ROC and alternative FROC curves were used as laboratory figures of merit. Bootstrap analysis was conducted to estimate conventional agreement measures between laboratory and clinical figures of merit. Also computed was a pseudovalue-based image-level correctness measure of the laboratory interpretations, whose association with C as measured by the area (rAUC) under an appropriately defined relevance ROC curve, is as a measure of the clinical relevance of a laboratory paradigm.

RESULTS

Low correlations (e.g. κ=0.244) and near chance level rAUC values (e.g. 0.598), attributable to differences between the clinical and laboratory paradigms, were observed. The absolute width of the confidence interval was 0.38 for the interparadigm differences of the conventional measures and 0.14 for the difference of the rAUCs.

CONCLUSION

The rAUC measure was consistent with the traditional measures but was more sensitive to the differences in clinical relevance. A new relevance ROC method for quantifying the clinical relevance of a laboratory paradigm is proposed.

A status report on free-response analysis

The purpose of this paper is to summarise recent progress in free-response receiver operating characteristic (FROC) methodology. These are: (1) jackknife alternative FROC analysis including recent extensions and alternative methods; (2) the search-model simulator that enables validation and objective comparison of different methods of analysing the data; (3) case-based analysis that has the potential of greater clinical relevance than conventional free-response analysis; (4) a method for collectively analysing the multiple lesion types in an image (e.g. microcalcifications, masses and architectural distortions); (5) a method for sample-size estimation for FROC studies; and (6) a method for determining an objective proximity criterion, namely how close must a mark be to a true lesion in order to credit the observer for a true localisation. FROC analysis is being increasingly used to evaluate the imaging systems and understanding of recent progress should help researchers conduct better FROC studies.

Consistency of methods for analysing location-specific data

Although the receiver operating characteristic (ROC) method is the acknowledged gold-standard for imaging system assessment, it ignores localisation information and differentiation between multiple abnormalities per case. As the free-response ROC (FROC) method uses localisation information and more closely resembles the clinical reporting process, it is being increasingly used. A number of methods have been proposed to analyse the data that result from an FROC study: jackknife alternative FROC (JAFROC) and a variant termed JAFROC1, initial detection and candidate analysis (IDCA) and ROC analysis via the reduction of the multiple ratings on a case to a single rating. The focus of this paper was to compare JAFROC1, IDCA and the ROC analysis methods using a clinical FROC human data set. All methods agreed on the ordering of the modalities and all yielded statistically significant differences of the figures-of-merit, i.e. p < 0.05. Both IDCA and JAFROC1 yielded much smaller p-values than ROC. The results are consistent with a recent simulation-based validation study comparing these and other methods. In conclusion, IDCA or JAFROC1 analysis of FROC human data may be superior at detecting modality differences than ROC analysis.

Clinical relevance of the ROC and free-response paradigms for comparing imaging system efficacies

Observer performance studies are widely used to assess medical imaging systems. Unlike technical/engineering measurements observer performance include the entire imaging chain and the radiologist. However, the widely used receiver operating characteristic (ROC) method ignores lesion localisation information. The free-response ROC (FROC) method uses the location information to appropriately reward or penalise correct or incorrect localisations, respectively. This paper describes a method for improving the clinical relevance of FROC studies. The method consists of assigning appropriate risk values to the different lesions that may be present on a single image. A high-risk lesion is one that is critical to detect and act upon, and is assigned a higher risk value than a low-risk lesion, one that is relatively innocuous. Instead of simply counting the number of lesions that are detected, as is done in conventional FROC analysis, a risk-weighted count is used. This has the advantage of rewarding detections of high-risk lesions commensurately more than detections of lower risk lesions. Simulations were used to demonstrate that the new method, termed case-based analysis, results in a higher figure of merit for an expert who detects more high-risk lesions than a naive observer who detects more low-risk lesions, even though both detect the same total number of lesions. Conventional free-response analysis is unable to distinguish between the two types of observers. This paper also comments on the issue of clinical relevance of ROC analysis vs. FROC for tasks that involve lesion localisation.

Plasma levels of lipid peroxides in patients with Parkinson's disease

Oxidative stress is implicated as a major factor for nigral neuronal cell death. Metabolic failure in antioxidant mechanisms could hypothetically facilitate the chemical processes that lead to lipid peroxidation. To elucidate whether elevated lipid peroxidation rates might increase risk of developing Parkinson's disease (PD), the Authors determined plasma levels of malondialdehyde (MDA) in 80 PD patients and 80 controls. There was a significant difference between the plasma MDA levels of PD patients and controls (7.48 +/- 1.55 vs 5.1 +/- 1.26 nmol/ml). Plasma MDA levels were inversely related to the age of the PD patients (r = -0.46; p < 0.01) and age of onset but in the control group, no such correlation was observed between the plasma MDA and age. However, there was no significant correlation between plasma MDA levels and the duration of disease, Hoehn and Yahr stages and the Unified Parkinson's Disease Rating Scale (UPDRS). Thus, the results suggest that high plasma lipid peroxidation rates might contribute as a risk factor for PD in West Bengal.

Operating characteristics predicted by models for diagnostic tasks involving lesion localization

In 1996 Swensson published an observer model that predicted receiver operating characteristic (ROC), localization ROC (LROC), free-response ROC (FROC) and alternative FROC (AFROC) curves, thereby achieving "unification" of different observer performance paradigms. More recently a model termed initial detection and candidate analysis (IDCA) has been proposed for fitting computer aided detection (CAD) generated FROC data, and recently a search model for human observer FROC data has been proposed. The purpose of this study was to derive IDCA and the search model based expressions for operating characteristics, and to compare the predictions to the Swensson model. For three out of four mammography CAD data sets all models yielded good fits in the high-confidence region, i.e., near the lower end of the plots. The search model and IDCA tended to better fit the data in the low-confidence region, i.e., near the upper end of the plots, particularly for FROC curves for which the Swensson model predictions departed markedly from the data. For one data set none of the models yielded satisfactory fits. A unique characteristic of search model and IDCA predicted operating characteristics is that the operating point is not allowed to move continuously to the lowest confidence limit of the corresponding Swensson model curves. This prediction is actually observed in the CAD raw data and it is the primary reason for the poor FROC fits of the Swensson model in the low-confidence region.

Dose reduction and its influence on diagnostic accuracy and radiation risk in digital mammography: an observer performance study using an anthropomorphic breast phantom

This study aimed to investigate the effect of dose reduction on diagnostic accuracy and radiation risk in digital mammography. Simulated masses and microcalcifications were positioned in an anthropomorphic breast phantom. Thirty digital images, 14 with lesions, 16 without, were acquired of the phantom using a Mammomat Novation (Siemens, Erlangen, Germany) at each of three dose levels. These corresponded to 100%, 50% and 30% of the normally used average glandular dose (AGD; 1.3 mGy for a standard breast). Eight observers interpreted the 90 unprocessed images in a free response study, and the data were analysed with the jackknife free response receiver operating characteristic (JAFROC) method. Observer performance was assessed using the JAFROC figure of merit (FOM). The benefit of radiation risk reduction was estimated based on several risk models. There was no statistically significant difference in performance, as described by the FOM, between the 100% and the 50% dose levels. However, the FOMs for both the 100% and the 50% dose were significantly different from the corresponding quantity for the 30% dose level (F-statistic = 4.95, p-value = 0.01). A dose reduction of 50% would result in three to nine fewer breast cancer fatalities per 100,000 women undergoing annual screening from the age of 40 to 49 years. The results of the study indicate a possibility of reducing the dose to the breast to half the dose level currently used. This has to be confirmed in clinical studies, and possible differences depending on lesion type should be examined further.

ROC curves predicted by a model of visual search

In imaging tasks where the observer is uncertain whether lesions are present, and where they could be present, the image is searched for lesions. In the free-response paradigm, which closely reflects this task, the observer provides data in the form of a variable number of mark-rating pairs per image. In a companion paper a statistical model of visual search has been proposed that has parameters characterizing the perceived lesion signal-to-noise ratio, the ability of the observer to avoid marking non-lesion locations, and the ability of the observer to find lesions. The aim of this work is to relate the search model parameters to receiver operating characteristic (ROC) curves that would result if the observer reported the rating of the most suspicious finding on an image as the overall rating. Also presented are the probability density functions (pdfs) of the underlying latent decision variables corresponding to the highest rating for normal and abnormal images. The search-model-predicted ROC curves are 'proper' in the sense of never crossing the chance diagonal and the slope is monotonically changing. They also have the interesting property of not allowing the observer to move the operating point continuously from the origin to (1, 1). For certain choices of parameters the operating points are predicted to be clustered near the initial steep region of the curve, as has been observed by other investigators. The pdfs are non-Gaussians, markedly so for the abnormal images and for certain choices of parameter values, and provide an explanation for the well-known observation that experimental ROC data generally imply a wider pdf for abnormal images than for normal images. Some features of search-model-predicted ROC curves and pdfs resemble those predicted by the contaminated binormal model, but there are significant differences. The search model appears to provide physical explanations for several aspects of experimental ROC curves.

A search model and figure of merit for observer data acquired according to the free-response paradigm

Search is a basic activity that is performed routinely in many different tasks. In the context of medical imaging it involves locating lesions in images under conditions of uncertainty regarding the number and locations of lesions that may be present. A search model is presented that applies to situations, as in the free-response paradigm, where on each image the number of normal regions that could be mistaken for lesions is unknown, and the number of observer generated localizations of suspicious regions (marks) is unpredictable. The search model is based on a two-stage model that has been proposed in the literature, according to which, at the first stage (the preattentive stage) the observer uses mainly peripheral vision to identify likely lesion candidates, and at the second stage the observer decides (i.e., cognitively evaluates) whether or not to report the candidates. The search model regards the unpredictable numbers of lesion and non-lesion localizations as random variables and models them via appropriate statistical distributions. The model has three parameters quantifying the lesion signal-to-noise ratio, the observer's expertise at rejecting non-lesion locations, and the observer's expertise at finding lesions. A figure-of-merit quantifying the observer's search performance is described. The search model bears a close resemblance to the initial detection and candidate analysis (IDCA) model that has been recently proposed for analysing computer aided detection (CAD) algorithms. The ability to analytically model and quantify the search process would enable more powerful assessment and optimization of performance in these activities, which could be highly significant.

Breast tissue density quantification via digitized mammograms

Studies reported in the literature indicate that breast cancer risk is associated with mammographic densities. An objective, repeatable, and a quantitative measure of risk derived from mammographic densities will be of considerable use in recommending alternative screening paradigms and/or preventive measures. However, image processing efforts toward this goal seem to be sparse in the literature, and automatic and efficient methods do not seem to exist. In this paper, we describe and validate an automatic and reproducible method to segment dense tissue regions from fat within breasts from digitized mammograms using scale-based fuzzy connectivity methods. Different measures for characterizing mammographic density are computed from the segmented regions and their robustness in terms of their linear correlation across two different projections--cranio-caudal and medio-lateral-oblique--are studied. The accuracy of the method is studied by computing the area of mismatch of segmented dense regions using the proposed method and using manual outlining. A comparison between the mammographic density parameter taking into account the original intensities and that just considering the segmented area indicates that the former may have some advantages over the latter.

Renal cyst pseudoenhancement: beam-hardening effects on CT numbers

PURPOSE

To determine if simple renal cysts may be accurately characterized with helical computed tomography (CT) during peak levels of renal enhancement.

MATERIALS AND METHODS

Water-filled "cysts" were suspended in varying concentrations of iodine solution, meant to simulate varying levels of renal enhancement, within an abdominal phantom. Volume-averaging effects were minimized by scanning cylindric 5-30-mm cysts with a helical technique (collimation, 5 mm; pitch, 1:1). Axial and helical techniques were then compared, and volume-averaging effects were evaluated by scanning 10- and 20-mm round cysts with 3-, 5-, and 7-mm collimation at background attenuation levels of 100 and 200 HU.

RESULTS

Cylindric cyst attenuation increased consistently with increasing background attenuation. As background attenuation increased by 90 HU, attenuation increased by 11-17 HU in small (5- or 10-mm) cysts, and by 7-9 HU in large (15-30-mm) cysts. As background attenuation increased by 180 HU, attenuation increased by 18-28 HU in small cysts and by 10-15 HU in large cysts. Spherical cyst attenuation differences were maximized when smaller cysts were imaged with larger collimation, which is when volume-averaging effects became apparent. Axial and helical CT numbers did not differ substantially. Computer simulation studies showed that the observed effect could not be explained by beam hardening alone.

CONCLUSION

Pseudoenhancement of renal cysts may occur if helical CT is performed during peak renal enhancement. CT algorithm modification may be necessary to correct for this effect, which is likely related to an inadequate algorithmic correction for beam hardening.

The effect of the antiscatter grid on full-field digital mammography phantom images

Computer Analysis of Mammography Phantom Images (CAMPI) is a method for making quantitative measurements of image quality. This article reports on a recent application of this method to a prototype full-field digital mammography (FFDM) machine. Images of a modified ACR phantom were acquired on the General Electric Diagnostic Molybdenum Rhodium (GE-DMR) FFDM machine at a number of x-ray techniques, both with and without the scatter reduction grid. The techniques were chosen so that one had sets of grid and non-grid images with matched doses (200 mrads) and matched gray-scale values (1500). A third set was acquired at constant 26 kVp and varying mAs for both grid conditions. Analyses of the images yielded signal-to-noise-ratio (SNR), contrast and noise corresponding to each target object, and a non-uniformity measure. The results showed that under conditions of equal gray-scale value the grid images were markedly superior, albeit at higher doses than the non-grid images. Under constant dose conditions, the non-grid images were slightly superior in SNR (7%) but markedly less uniform (60%). Overall, the grid images had substantially greater contrast and superior image uniformity. These conclusions applied to the whole kVp range studied for the Mo-Mo target filter combination and 4 cm of breast equivalent material of average composition. These results suggest that use of the non-grid technique in digital mammography with the GE-DMR-FFDM unit, is presently not warranted. With improved uniformity correction procedure, this conclusion would change and one should be able to realize a 14% reduction in patient dose at the same SNR by using a non-grid technique.

Computer analysis of mammography phantom images (CAMPI): an application to the measurement of microcalcification image quality of directly acquired digital images

The purpose of this investigation was to apply the recently developed CAMPI (computer analysis of mammography phantom images) method to a Fischer Mammotest Stereotactic Digital Biopsy machine. Another aim was to further elucidate the nature of the empirically introduced CAMPI measures. Images of an American College of Radiology (ACR) accreditation phantom centered on the largest two speck groups were obtained on this machine under a variety of x-ray conditions. An additional measure, alternative SNR (ASNR) is introduced which is complementary to the SNR measure. Analyses of the Mammotest images revealed that the mAs and kVp dependencies of the CAMPI measures could be understood from basic imaging physics principles. It is shown that: (1) the measures reflect the expected linearity of the digital detector and Poisson photon statistics; (2) under automatic exposure control (AEC) conditions the signal (SIG) measure is proportional to subject contrast; and (3) under AEC conditions the noise (NOI) measure is proportional to the square root of the average absorbed photon energy. Correspondence with basic imaging physics principles shows that the measures are significantly free of artifacts. Precision of the CAMPI measures exceeds that of human observers by orders of magnitude. CAMPI measures are expected to be more relevant to clinical mammography than Fourier metrics as the measurements are done on objects of arbitrary shape and size that were designed by the manufacturer to resemble various detection tasks in mammography. It is concluded that CAMPI can perform objective and highly precise evaluations of phantom image quality in mammography. It could be used as a sophisticated quality control tool, as a replacement for the current ACR/MQSA phantom evaluation program, and to evaluate the rapidly evolving digital mammography technology.

Vitiligo, psoralen, and melanogenesis: some observations and understanding

Since the etiology of vitiligo is still unknown, we searched for some abnormal biochemical parameters, if any, in subjects with vitiligo. Higher urinary excretion of indole metabolites in vitiliginous patients have been noted, in association with higher dioxygenase, superoxide dismutase, and tyrosine aminotransferase activity in their serum. Similar results have also been found in an animal model, Bufo melanostictus, during induced tyrosinase inhibition. Treatment with psoralen can reverse the parameters, except tyrosine aminotransferase, to a normal level. Although psoralens are not the magic bullet for the therapy of vitiligo, they are still being used as a chemotherapeutic agent against vitiligo on a major scale to date. Tryptophan was found to participate in the pathway of melanogenesis, as a precursor as well as a positive regulator of tyrosinase. Its behavior in this regard is much more similar to the conventional substrates tyrosine and dopa (dihydroxyphenylalanine). In consideration of combined participation of tyrosine and tryptophan in the synthesis of melanin and its breakdown, the possible influence of different enzymatic reactions, like mono-oxygenase, dioxygenase, and deamination, has been suggested.

A ROC study of AMBER and conventional chest imaging in the detection of simulated interstitial lung disease

PURPOSE

The aim of this study was to compare the detection of simulated interstitial lung disease with Advanced Multiple Beam Equalization Radiography (AMBER) and conventional wide latitude screen-film radiography.

MATERIALS AND METHODS

Interstitial disease of varying severity was simulated with overlays on an anthropomorphic chest phantom. A total of 60 images per modality was used in a Receiver Operating Characteristic (ROC) study.

RESULTS

AMBER performed significantly better than conventional radiography for all readers (P < 0.02). The difference was even more significant for the radiologist readers (P = 0.001). In each case the difference in ROC areas was between 5% and 9%.

CONCLUSION

AMBER is superior to conventional wide latitude screen-film imaging in detecting the subtle patterns used to simulate interstitial lung disease.

Quantitative versus subjective evaluation of mammography accreditation phantom images

The precision of quantitative and subjective evaluations of phantom image quality has been studied. Twenty-seven images of the American College of Radiology (ACR) mammography accreditation phantom were acquired under different x-ray techniques and digitized. Several quantitative image quality measures were obtained from each image by analyzing microcalcification and nodule target objects in the phantom. All images were also scored subjectively by 8 observers, each of whom provided a count of the number of objects seen in each target class (fibrils, microcalcifications, and nodules). An analysis was performed to predict the subjective measurements from the quantitative measurements and to estimate their variabilities. It was found that the subjective measures could be well predicted by the quantitative measures and that the variance of the quantitative measures was significantly smaller than that of the subjective measure, by almost a factor of 10. The implication for the ACR accreditation program for mammography is that a substantial improvement is possible in the image quality evaluation process by performing computerized analysis of the phantom images in addition to subjective analysis.