New Theoretical Results on Channelized Hotelling Observer Performance Estimation with Known Difference of Class Means

Accurate and efficient measurement of channelized Hotelling observer-based low-contrast detectability on the ACR CT accreditation phantom

BACKGROUND

Current CT quality control (QC) for low-contrast detectability relies on visual inspection and measurement of contrast-to-noise ratio (CNR). However, CNR numbers become unreliable when it comes to nonlinear methods, such as iterative reconstruction (IR) and deep-learning-based techniques. Image quality metrics using channelized Hotelling observer (CHO) have been validated to be well correlated with human observer performance on phantom-based and patient-based tasks, but it has not been widely used in routine CT QC mainly because the CHO calculation typically requires a large number of repeated scans in order to provide accurate and precise estimate of index of detectability (d').

PURPOSE

The main goal of this work is to optimize channel filters and other CHO parameters and accurately estimate the low-contrast detectability with minimum number of repeated scans for the widely used American College of Radiology (ACR) CT accreditation phantom so that it can become practically feasible for routine CT QC tests.

METHODS

To provide a converged d' value, an ACR phantom was repeatedly scanned 100 times at three dose levels (24, 12, and 6 mGy). Images were reconstructed with two kernels (FBP Br44 and IR Br44-3). d' as a function of number of repeated scans was determined for different number of background regions of interest (ROIs), different number of low-contrast objects, different number of slices per each object, and different channel filter options. A reference d' was established using the optimized CHO setting, and the bias of d' was quantified using the d' calculated from all 100 repeated scans. The variation of d' at each condition was estimated using a resampling method combining random subsampling among 100 repeated scans and bootstrapping of the ensembles of signal and background ROIs.

RESULTS

Optimized parameters in CHO calculation were determined: two background ROIs per object, four objects per low-contrast object size, nine non-overlapping slices per object, and a 4-channel Gabor filter. The bias and uncertainty were estimated at different numbers of repeated scans using these parameters. When only one single scan was used in the CHO calculation, the bias of d' was below 6.2% and the uncertainty 15.6-19.6% for the 6, 5, and 4 mm objects, while with three repeated scans the bias was below 2.0% and uncertainty 8.7-10.9% for the three object sizes.

CONCLUSION

With optimized parameter settings in CHO, efficient and accurate measurement of low-contrast detectability on the commonly used ACR phantom becomes feasible, which could potentially lead to adoption of CHO-based low-contrast evaluation in routine QC tests.

Assessment of collimators in radium-223 imaging with channelized Hotelling observer: a simulation study

OBJECTIVE

Radium-223 (²²³Ra) is used in unsealed radionuclide therapy for metastatic bone tumors. The aim of this study is to apply a computational model observer to ²²³Ra planar images, and to assess the performance of collimators in ²²³Ra imaging.

METHODS

The ²²³Ra planar images were created via an in-house Monte Carlo simulation code using HEXAGON and NAI modules. The phantom was a National Electrical Manufacturers Association body phantom with a hot sphere. The concentration of the background was 55 Bq/mL, and the sphere was approximately 1.5-20 times that of the background concentration. The acquisition time was 10 min. The photopeaks (and the energy window) were 84 (full width of energy window: 20%), 154 (15%), and 270 keV (10%). Each 40 images, with and without hot concentration, were applied to a three-channel difference-of-Gaussian channelized Hotelling observer (CHO), and the signal-to-noise ratio (SNR) of the hot region was calculated. The images were examined using five different collimators: two low-energy general-purpose (LEGP), two medium-energy general-purpose (MEGP), and one high-energy general-purpose (HEGP) collimators.

RESULTS

The SNR value was linearly proportional to the contrast of the hot region for all collimators and energy windows. The images of the 84-keV energy window with the MEGP collimator that have thicker septa and larger holes produced the highest SNR value. The SNR values of two LEGP collimators were approximately half of the MEGP collimators. The HEGP collimator was halfway between the MEGP and LEGP. Similar characteristics were observed for other energy windows (154, 270 keV). The SNR value of images captured via the 270-keV energy window was larger than 154-keV, although the sensitivity of the 270-keV energy window is lower than 154-keV. The results suggested a positive correlation between the SNR value and the fraction of unscattered photons.

CONCLUSIONS

The SNR value of CHO reflected the performance of collimators and was available to assess and quantitatively evaluate the collimator performance in ²²³Ra imaging. The SNR value depends on the magnitudes of unscattered photon count and the fraction of unscattered photon count. Consequently, in this study, MEGP collimators performed better than LEGP and HEGP collimators for ²²³Ra imaging.

Factors affecting the normality of channel outputs of channelized model observers: an investigation using realistic myocardial perfusion SPECT images

The channelized Hotelling observer (CHO) uses the first- and second-order statistics of channel outputs under both hypotheses to compute test statistics used in binary classification tasks. If these input data deviate from a multivariate normal (MVN) distribution, the classification performance will be suboptimal compared to an ideal observer operating on the same channel outputs. We conducted a comprehensive investigation to rigorously study the validity of the MVN assumption under various kinds of background and signal variability in a realistic population of phantoms. The study was performed in the context of myocardial perfusion SPECT imaging; anatomical, uptake (intensity), and signal variability were simulated. Quantitative measures and graphical approaches applied to the outputs of each channel were used to investigate the amount and type of deviation from normality. For some types of background and signal variations, the channel outputs, under both hypotheses, were non-normal (i.e., skewed or multimodal). This indicates that, for realistic medical images in cases where there is signal or background variability, the normality of the channel outputs should be evaluated before applying a CHO. Finally, the different degrees of departure from normality of the various channels are explained in terms of violations of the central limit theorem.

Exact confidence intervals for channelized Hotelling observer performance in image quality studies

Task-based assessments of image quality constitute a rigorous, principled approach to the evaluation of imaging system performance. To conduct such assessments, it has been recognized that mathematical model observers are very useful, particularly for purposes of imaging system development and optimization. One type of model observer that has been widely applied in the medical imaging community is the channelized Hotelling observer (CHO), which is well-suited to known-location discrimination tasks. In the present work, we address the need for reliable confidence interval estimators of CHO performance. Specifically, we show that the bias associated with point estimates of CHO performance can be overcome by using confidence intervals proposed by Reiser for the Mahalanobis distance. In addition, we find that these intervals are well-defined with theoretically-exact coverage probabilities, which is a new result not proved by Reiser. The confidence intervals are tested with Monte Carlo simulation and demonstrated with two examples comparing X-ray CT reconstruction strategies. Moreover, commonly-used training/testing approaches are discussed and compared to the exact confidence intervals. MATLAB software implementing the estimators described in this work is publicly available at http://code.google.com/p/iqmodelo/.

Region of interest based Hotelling observer for computed tomography with comparison to alternative methods

We compare several approaches to estimation of Hotelling observer (HO) performance in x-ray computed tomography (CT). We consider the case where the signal of interest is small so that the reconstructed image can be restricted to a small region of interest (ROI) surrounding the signal. This reduces the dimensionality of the image covariance matrix so that direct computation of HO metrics within the ROI is feasible. We propose that this approach is directly applicable to systems optimization in CT; however, many alternative approaches exist, which make computation of HO performance tractable through a range of approximations, assumptions, or estimation strategies. Here, we compare several of these methods, including the use of Laguerre-Gauss channels, discrete Fourier domain computation of the HO (which assumes noise stationarity), and two approaches to HO estimation through samples of noisy images. Since our method computes HO performance exactly within an ROI, this allows us to investigate the validity of the assumptions inherent in various common approaches to HO estimation, such as the stationarity assumption in the case of the discrete Fourier transform domain method.

Computational observers and visualization methods for stereoscopic medical imaging

As stereoscopic display devices become common, their image quality assessment evaluation becomes increasingly important. Most studies conducted on 3D displays are based on psychophysics experiments with humans rating their experience based on detection tasks. The physical measurements do not map to effects on signal detection performance. Additionally, human observer study results are often subjective and difficult to generalize. We designed a computational stereoscopic observer approach inspired by the mechanisms of stereopsis in human vision for task-based image assessment that makes binary decisions based on a set of image pairs. The stereo-observer is constrained to a left and a right image generated using a visualization operator to render voxel datasets. We analyze white noise and lumpy backgrounds using volume rendering techniques. Our simulation framework generalizes many different types of model observers including existing 2D and 3D observers as well as providing flexibility to formulate a stereo model observer approach following the principles of stereoscopic viewing. This methodology has the potential to replace human observer studies when exploring issues with stereo display devices to be used in medical imaging. We show results quantifying the changes in performance when varying stereo angle as measured by an ideal linear stereoscopic observer. Our findings indicate that there is an increase in performance of about 13-18% for white noise and 20-46% for lumpy backgrounds, where the stereo angle is varied from 0 to 30. The applicability of this observer extends to stereoscopic displays used for in the areas of medical and entertainment imaging applications.

Task-based detectability in CT image reconstruction by filtered backprojection and penalized likelihood estimation

PURPOSE

Nonstationarity is an important aspect of imaging performance in CT and cone-beam CT (CBCT), especially for systems employing iterative reconstruction. This work presents a theoretical framework for both filtered-backprojection (FBP) and penalized-likelihood (PL) reconstruction that includes explicit descriptions of nonstationary noise, spatial resolution, and task-based detectability index. Potential utility of the model was demonstrated in the optimal selection of regularization parameters in PL reconstruction.

METHODS

Analytical models for local modulation transfer function (MTF) and noise-power spectrum (NPS) were investigated for both FBP and PL reconstruction, including explicit dependence on the object and spatial location. For FBP, a cascaded systems analysis framework was adapted to account for nonstationarity by separately calculating fluence and system gains for each ray passing through any given voxel. For PL, the point-spread function and covariance were derived using the implicit function theorem and first-order Taylor expansion according to Fessler ["Mean and variance of implicitly defined biased estimators (such as penalized maximum likelihood): Applications to tomography," IEEE Trans. Image Process. 5(3), 493-506 (1996)]. Detectability index was calculated for a variety of simple tasks. The model for PL was used in selecting the regularization strength parameter to optimize task-based performance, with both a constant and a spatially varying regularization map.

RESULTS

Theoretical models of FBP and PL were validated in 2D simulated fan-beam data and found to yield accurate predictions of local MTF and NPS as a function of the object and the spatial location. The NPS for both FBP and PL exhibit similar anisotropic nature depending on the pathlength (and therefore, the object and spatial location within the object) traversed by each ray, with the PL NPS experiencing greater smoothing along directions with higher noise. The MTF of FBP is isotropic and independent of location to a first order approximation, whereas the MTF of PL is anisotropic in a manner complementary to the NPS. Task-based detectability demonstrates dependence on the task, object, spatial location, and smoothing parameters. A spatially varying regularization "map" designed from locally optimal regularization can improve overall detectability beyond that achievable with the commonly used constant regularization parameter.

CONCLUSIONS

Analytical models for task-based FBP and PL reconstruction are predictive of nonstationary noise and resolution characteristics, providing a valuable framework for understanding and optimizing system performance in CT and CBCT.

Model observers in medical imaging research

Model observers play an important role in the optimization and assessment of imaging devices. In this review paper, we first discuss the basic concepts of model observers, which include the mathematical foundations and psychophysical considerations in designing both optimal observers for optimizing imaging systems and anthropomorphic observers for modeling human observers. Second, we survey a few state-of-the-art computational techniques for estimating model observers and the principles of implementing these techniques. Finally, we review a few applications of model observers in medical imaging research.