Task-based measures of image quality and their relation to radiation dose and patient risk

An image quality assessment index based on image features and keypoints for X-ray CT images

Optimization tasks in diagnostic radiological imaging require objective quantitative metrics that correlate with the subjective perception of observers. However, although one such metric, the structural similarity index (SSIM), is popular, it has limitations across various aspects in its application to medical images. In this study, we introduce a novel image quality evaluation approach based on keypoints and their associated unique image feature values, focusing on developing a framework to address the need for robustness and interpretability that are lacking in conventional methodologies. The proposed index quantifies and visualizes the distance between feature vectors associated with keypoints, which varies depending on changes in the image quality. This metric was validated on images with varying noise levels and resolution characteristics, and its applicability and effectiveness were examined by evaluating images subjected to various affine transformations. In the verification of X-ray computed tomography imaging using a head phantom, the distances between feature descriptors for each keypoint increased as the image quality degraded, exhibiting a strong correlation with the changes in the SSIM. Notably, the proposed index outperformed conventional full-reference metrics in terms of robustness to various transformations which are without changes in the image quality. Overall, the results suggested that image analysis performed using the proposed framework could effectively visualize the corresponding feature points, potentially harnessing lost feature information owing to changes in the image quality. These findings demonstrate the feasibility of applying the novel index to analyze changes in the image quality. This method may overcome limitations inherent in conventional evaluation methodologies and contribute to medical image analysis in the broader domain.

Task-based transferable deep-learning scatter correction in cone beam computed tomography: a simulation study

Purpose

X-ray scatter significantly affects the image quality of cone beam computed tomography (CBCT). Although convolutional neural networks (CNNs) have shown promise in correcting x-ray scatter, their effectiveness is hindered by two main challenges: the necessity for extensive datasets and the uncertainty regarding model generalizability. This study introduces a task-based paradigm to overcome these obstacles, enhancing the application of CNNs in scatter correction.

Approach

Using a CNN with U-net architecture, the proposed methodology employs a two-stage training process for scatter correction in CBCT scans. Initially, the CNN is pre-trained on approximately 4000 image pairs from geometric phantom projections, then fine-tuned using transfer learning (TL) on 250 image pairs of anthropomorphic projections, enabling task-specific adaptations with minimal data. 2D scatter ratio (SR) maps from projection data were considered as CNN targets, and such maps were used to perform the scatter prediction. The fine-tuning process for specific imaging tasks, like head and neck imaging, involved simulating scans of an anthropomorphic phantom and pre-processing the data for CNN retraining.

Results

For the pre-training stage, it was observed that SR predictions were quite accurate (SSIM ≥ 0.9 ). The accuracy of SR predictions was further improved after TL, with a relatively short retraining time (≈ 70 times faster than pre-training) and using considerably fewer samples compared to the pre-training dataset (≈ 12 times smaller).

Conclusions

A fast and low-cost methodology to generate task-specific CNN for scatter correction in CBCT was developed. CNN models trained with the proposed methodology were successful to correct x-ray scatter in anthropomorphic structures, unknown to the network, for simulated data.

Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data

Reliable performance of PET segmentation algorithms on clinically relevant tasks is required for their clinical translation. However, these algorithms are typically evaluated using figures of merit (FoMs) that are not explicitly designed to correlate with clinical task performance. Such FoMs include the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the Hausdorff distance (HD). The objective of this study was to investigate whether evaluating PET segmentation algorithms using these task-agnostic FoMs yields interpretations consistent with evaluation on clinically relevant quantitative tasks. Methods: We conducted a retrospective study to assess the concordance in the evaluation of segmentation algorithms using the DSC, JSC, and HD and on the tasks of estimating the metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumors from PET images of patients with non-small cell lung cancer. The PET images were collected from the American College of Radiology Imaging Network 6668/Radiation Therapy Oncology Group 0235 multicenter clinical trial data. The study was conducted in 2 contexts: (1) evaluating conventional segmentation algorithms, namely those based on thresholding (SUVmax40% and SUVmax50%), boundary detection (Snakes), and stochastic modeling (Markov random field-Gaussian mixture model); (2) evaluating the impact of network depth and loss function on the performance of a state-of-the-art U-net-based segmentation algorithm. Results: Evaluation of conventional segmentation algorithms based on the DSC, JSC, and HD showed that SUVmax40% significantly outperformed SUVmax50%. However, SUVmax40% yielded lower accuracy on the tasks of estimating MTV and TLG, with a 51% and 54% increase, respectively, in the ensemble normalized bias. Similarly, the Markov random field-Gaussian mixture model significantly outperformed Snakes on the basis of the task-agnostic FoMs but yielded a 24% increased bias in estimated MTV. For the U-net-based algorithm, our evaluation showed that although the network depth did not significantly alter the DSC, JSC, and HD values, a deeper network yielded substantially higher accuracy in the estimated MTV and TLG, with a decreased bias of 91% and 87%, respectively. Additionally, whereas there was no significant difference in the DSC, JSC, and HD values for different loss functions, up to a 73% and 58% difference in the bias of the estimated MTV and TLG, respectively, existed. Conclusion: Evaluation of PET segmentation algorithms using task-agnostic FoMs could yield findings discordant with evaluation on clinically relevant quantitative tasks. This study emphasizes the need for objective task-based evaluation of image segmentation algorithms for quantitative PET.

Ideal Observer Computation by Use of Markov-Chain Monte Carlo With Generative Adversarial Networks

Medical imaging systems are often evaluated and optimized via objective, or task-specific, measures of image quality (IQ) that quantify the performance of an observer on a specific clinically-relevant task. The performance of the Bayesian Ideal Observer (IO) sets an upper limit among all observers, numerical or human, and has been advocated for use as a figure-of-merit (FOM) for evaluating and optimizing medical imaging systems. However, the IO test statistic corresponds to the likelihood ratio that is intractable to compute in the majority of cases. A sampling-based method that employs Markov-chain Monte Carlo (MCMC) techniques was previously proposed to estimate the IO performance. However, current applications of MCMC methods for IO approximation have been limited to a small number of situations where the considered distribution of to-be-imaged objects can be described by a relatively simple stochastic object model (SOM). As such, there remains an important need to extend the domain of applicability of MCMC methods to address a large variety of scenarios where IO-based assessments are needed but the associated SOMs have not been available. In this study, a novel MCMC method that employs a generative adversarial network (GAN)-based SOM, referred to as MCMC-GAN, is described and evaluated. The MCMC-GAN method was quantitatively validated by use of test-cases for which reference solutions were available. The results demonstrate that the MCMC-GAN method can extend the domain of applicability of MCMC methods for conducting IO analyses of medical imaging systems.

Unsharp Structure Guided Filtering for Self-Supervised Low-Dose CT Imaging

Low-dose computed tomography (LDCT) imaging faces great challenges. Although supervised learning has revealed great potential, it requires sufficient and high-quality references for network training. Therefore, existing deep learning methods have been sparingly applied in clinical practice. To this end, this paper presents a novel Unsharp Structure Guided Filtering (USGF) method, which can reconstruct high-quality CT images directly from low-dose projections without clean references. Specifically, we first employ low-pass filters to estimate the structure priors from the input LDCT images. Then, inspired by classical structure transfer techniques, deep convolutional networks are adopted to implement our imaging method which combines guided filtering and structure transfer. Finally, the structure priors serve as the guidance images to alleviate over-smoothing, as they can transfer specific structural characteristics to the generated images. Furthermore, we incorporate traditional FBP algorithms into self-supervised training to enable the transformation of projection domain data to the image domain. Extensive comparisons and analyses on three datasets demonstrate that the proposed USGF has achieved superior performance in terms of noise suppression and edge preservation, and could have a significant impact on LDCT imaging in the future.

Approximating the Hotelling observer with autoencoder-learned efficient channels for binary signal detection tasks

Purpose

The objective assessment of image quality (IQ) has been advocated for the analysis and optimization of medical imaging systems. One method of computing such IQ metrics is through a numerical observer. The Hotelling observer (HO) is the optimal linear observer, but conventional methods for obtaining the HO can become intractable due to large image sizes or insufficient data. Channelized methods are sometimes employed in such circumstances to approximate the HO. The performance of such channelized methods varies, with different methods obtaining superior performance to others depending on the imaging conditions and detection task. A channelized HO method using an AE is presented and implemented across several tasks to characterize its performance.

Approach

The process for training an AE is demonstrated to be equivalent to developing a set of channels for approximating the HO. The efficiency of the learned AE-channels is increased by modifying the conventional AE loss function to incorporate task-relevant information. Multiple binary detection tasks involving lumpy and breast phantom backgrounds across varying dataset sizes are considered to evaluate the performance of the proposed method and compare to current state-of-the-art channelized methods. Additionally, the ability of the channelized methods to generalize to images outside of the training dataset is investigated.

Results

AE-learned channels are demonstrated to have comparable performance with other state-of-the-art channel methods in the detection studies and superior performance in the generalization studies. Incorporating a cleaner estimate of the signal for the detection task is also demonstrated to significantly improve the performance of the proposed method, particularly in datasets with fewer images.

Conclusions

AEs are demonstrated to be capable of learning efficient channels for the HO. The resulting significant increase in detection performance for small dataset sizes when incorporating a signal prior holds promising implications for future assessments of imaging technologies.

Development and task-based evaluation of a scatter-window projection and deep learning-based transmission-less attenuation compensation method for myocardial perfusion SPECT

Attenuation compensation (AC) is beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). However, traditional AC methods require the availability of a transmission scan, most often a CT scan. This approach has the disadvantages of increased radiation dose, increased scanner cost, and the possibility of inaccurate diagnosis in cases of misregistration between the SPECT and CT images. Further, many SPECT systems do not include a CT component. To address these issues, we developed a Scatter-window projection and deep Learning-based AC (SLAC) method to perform AC without a separate transmission scan. To investigate the clinical efficacy of this method, we then objectively evaluated the performance of this method on the clinical task of detecting perfusion defects on MPI in a retrospective study with anonymized clinical SPECT/CT stress MPI images. The proposed method was compared with CT-based AC (CTAC) and no-AC (NAC) methods. Our results showed that the SLAC method yielded an almost overlapping receiver operating characteristic (ROC) plot and a similar area under the ROC (AUC) to the CTAC method on this task. These results demonstrate the capability of the SLAC method for transmission-less AC in SPECT and motivate further clinical evaluation.

A nonparametric measure of noise in x-ray diagnostic images-mammography

Objective.In x-ray diagnostics, modern image reconstruction or image processing methods may render established methods of image quality assessment inadequate. Task specific quality assessment by using model observers has the disadvantage of being very labour-intensive. Therefore, it appears highly desirable to develop novel image quality parameters that neither rely on the linearity and the shift-invariace of the imaging system nor require the acquisition of hundreds of images as is necessary for the application of model observers, and which can be derived directly from diagnostic images.Approach.A new measure for the noise based on non-maximum-suppression images is defined and its properties are explored using simulated images before it is applied to an exposure series of mammograms of a homogeneous phantom and a 3D-printed breast phantom to demonstrate its usefulness under realistic conditions.Main results.The new noise parameter cannot only be derived from images with a homogeneous background but it can be extracted directly from images containing anatomic structures and is proportional to the standard deviation of the noise. At present, the applicability is restricted to mammography, which satisfies the assumption of short covariance length of the noise.Significance.The new measure of the noise is but a first step of the development of a set of parameters that are required to quantify image quality directly from diagnostic images without relying on the assumption of a linear, shift-invariant system, e.g. by providing measures of sharpness, contrast and structural complexity, in addition to the noise measure. For mammography, a convenient method is now available to quantify noise in processed diagnostic images.

Development and task-based evaluation of a scatter-window projection and deep learning-based transmission-less attenuation compensation method for myocardial perfusion SPECT

Attenuation compensation (AC) is beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). However, traditional AC methods require the availability of a transmission scan, most often a CT scan. This approach has the disadvantage of increased radiation dose, increased scanner costs, and the possibility of inaccurate diagnosis in cases of misregistration between the SPECT and CT images. Further, many SPECT systems do not include a CT component. To address these issues, we developed a Scatter-window projection and deep Learning-based AC (SLAC) method to perform AC without a separate transmission scan. To investigate the clinical efficacy of this method, we then objectively evaluated the performance of this method on the clinical task of detecting perfusion defects on MPI in a retrospective study with anonymized clinical SPECT/CT stress MPI images. The proposed method was compared with CT-based AC (CTAC) and no-AC (NAC) methods. Our results showed that the SLAC method yielded an almost overlapping receiver operating characteristic (ROC) plot and a similar area under the ROC (AUC) to the CTAC method on this task. These results demonstrate the capability of the SLAC method for transmission-less AC in SPECT and motivate further clinical evaluation.

Need for objective task-based evaluation of AI-based segmentation methods for quantitative PET

Artificial intelligence (AI)-based methods are showing substantial promise in segmenting oncologic positron emission tomography (PET) images. For clinical translation of these methods, assessing their performance on clinically relevant tasks is important. However, these methods are typically evaluated using metrics that may not correlate with the task performance. One such widely used metric is the Dice score, a figure of merit that measures the spatial overlap between the estimated segmentation and a reference standard (e.g., manual segmentation). In this work, we investigated whether evaluating AI-based segmentation methods using Dice scores yields a similar interpretation as evaluation on the clinical tasks of quantifying metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumor from PET images of patients with non-small cell lung cancer. The investigation was conducted via a retrospective analysis with the ECOG-ACRIN 6668/RTOG 0235 multi-center clinical trial data. Specifically, we evaluated different structures of a commonly used AI-based segmentation method using both Dice scores and the accuracy in quantifying MTV/TLG. Our results show that evaluation using Dice scores can lead to findings that are inconsistent with evaluation using the task-based figure of merit. Thus, our study motivates the need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.

Evaluation of the quantitative performance of non-enhanced dual-energy CT X-map in detecting acute ischemic brain stroke: A model observer study using computer simulation

PURPOSE

A simulation study was performed to evaluate the quantitative performance of X-map images-derived from non-enhanced (NE) dual-energy computed tomography (DECT)-in detecting acute ischemic stroke (AIS) compared with that of NE-DECT mixed images.

METHODS

A virtual phantom, 150 mm in diameter, filled with tissues comprising various gray- and white-matter proportions was used to generate pairs of NE-head images at 80 kV and Sn150 kV at three dose levels (20, 40, and 60 mGy). The phantom included an inserted low-contrast object, 15 mm in diameter, with four densities (0%, 5%, 10%, and 15%) mimicking ischemic edema. Mixed and X-map images were generated from these sets of images and compared in terms of detectability of ischemic edema using a channelized Hotelling observer (CHO). The area under the curve (AUC) of the receiver operating characteristic that generated CHO for each condition was used as a figure of merit.

RESULTS

The AUCs of X-map images were always significantly higher than those of mixed images (P < 0.001). The improvement in AUC for X-map images compared with that for mixed images at edema densities was 9.2%-12.6% at 20 mGy, 10.1%-17.7% at 40 mGy, and 14.0%-19.4% at 60 mGy. At any edema density, X-map images at 20 mGy resulted in higher AUCs than mixed images acquired at any other dose level (P < 0.001), which corresponded to a 66% dose reduction on X-map images.

CONCLUSIONS

The simulation study confirmed that NE-DECT X-map images have superior capability of detecting AIS than NE-DECT mixed images.

Performance evaluation of digital breast tomosynthesis systems: physical methods and experimental data

Digital breast tomosynthesis (DBT) has become a well-established breast imaging technique, whose performance has been investigated in many clinical studies, including a number of prospective clinical trials. Results from these studies generally point to non-inferiority in terms of microcalcification detection and superior mass-lesion detection for DBT imaging compared to digital mammography (DM). This modality has become an essential tool in the clinic for assessment and ad-hoc screening but is not yet implemented in most breast screening programmes at a state or national level. While evidence on the clinical utility of DBT has been accumulating, there has also been progress in the development of methods for technical performance assessment and quality control of these imaging systems. DBT is a relatively complicated ‘pseudo-3D’ modality whose technical assessment poses a number of difficulties. This paper reviews methods for the technical performance assessment of DBT devices, starting at the component level in part one and leading up to discussion of system evaluation with physical test objects in part two. We provide some historical and basic theoretical perspective, often starting from methods developed for DM imaging. Data from a multi-vendor comparison are also included, acquired under the medical physics quality control protocol developed by EUREF and currently being consolidated by a European Federation of Organisations for Medical Physics working group. These data and associated methods can serve as a reference for the development of reference data and provide some context for clinical studies.

Masked Joint Bilateral Filtering via Deep Image Prior for Digital X-ray Image Denoising

Medical image denoising faces great challenges. Although deep learning methods have shown great potential, their efficiency is severely affected by millions of trainable parameters. The non-linearity of neural networks also makes them difficult to be understood. Therefore, existing deep learning methods have been sparingly applied to clinical tasks. To this end, we integrate known filtering operators into deep learning and propose a novel Masked Joint Bilateral Filtering (MJBF) via deep image prior for digital X-ray image denoising. Specifically, MJBF consists of a deep image prior generator and an iterative filtering block. The deep image prior generator produces plentiful image priors by a multi-scale fusion network. The generated image priors serve as the guidance for the iterative filtering block, which is utilized for the actual edge-preserving denoising. The iterative filtering block contains three trainable Joint Bilateral Filters (JBFs), each with only 18 trainable parameters. Moreover, a masking strategy is introduced to reduce redundancy and improve the understanding of the proposed network. Experimental results on the ChestX-ray14 dataset and real data show that the proposed MJBF has achieved superior performance in terms of noise suppression and edge preservation. Tests on the portability of the proposed method demonstrate that this denoising modality is simple yet effective, and could have a clinical impact on medical imaging in the future.

Ideal-Observer Computation with anthropomorphic phantoms using Markov chain Monte Carlo

In medical imaging, it is widely recognized that image quality should be objectively evaluated based on performance in clinical tasks. To evaluate performance in signal-detection tasks, the ideal observer (IO) is optimal but also challenging to compute in clinically realistic settings. Markov Chain Monte Carlo (MCMC)-based strategies have demonstrated the ability to compute the IO using pre-computed projections of an anatomical database. To evaluate image quality in clinically realistic scenarios, the observer performance should be measured for a realistic patient distribution. This implies that the anatomical database should also be derived from a realistic population. In this manuscript, we propose to advance the MCMC-based approach towards achieving these goals. We then use the proposed approach to study the effect of anatomical database size on IO computation for the task of detecting perfusion defects in simulated myocardial perfusion SPECT images. Our preliminary results provide evidence that the size of the anatomical database affects the computation of the IO.

Visual grading experiments and optimization in CBCT dental implantology imaging: preliminary application of integrated visual grading regression

This study uses a general formulation of integrated visual grading regression (IVGR) and applies it to cone beam computed tomography (CBCT) scan data related to anatomical landmarks for dental implantology. The aim was to assess and predict a minimum acceptable dose for diagnostic imaging and reporting. A skull phantom was imaged with a CBCT unit at various diagnostic exposures. Key anatomical landmarks within the images were independently reviewed by three trained observers. Each provided an overall image quality score. Statistical analysis was carried out to examine the acceptability of the images taken, using an IVGR analysis that was formulized as a three-stage protocol including defining an integrated score, development of an ordinal regression, and investigation of the possibility for dose reduction through estimated parameters. For a unit increase in the logarithm of radiation dose, the odds ratio that the integrated score for an image assessed by observers being rated in a higher category was 3.940 (95% confidence interval: 1.016-15.280). When assessed by the observers, the minimum dose required to achieve a 75% probability for an image to be classified as at least acceptable was 1346.91 mGy·cm² dose area product (DAP), a 31% reduction compared to the 1962 mGy·cm² DAP default dosage of the CBCT unit. The kappa values of the intra and inter-observer reliability indicated moderate agreements, while a discrepancy among observers was also identified because each, as expected, perceived visibility differently. The results of this work demonstrate the IVGR's predictive value of dose saving in the effort to reduce dose to patients while maintaining reportable diagnostic image quality.

Evaluation of Apparent Noise on CT Images Using Moving Average Filters

This study aims to devise a simple method for evaluating the magnitude of texture noise (apparent noise) observed on computed tomography (CT) images scanned at a low radiation dose and reconstructed using iterative reconstruction (IR) and deep learning reconstruction (DLR) algorithms, and to evaluate the apparent noise in CT images reconstructed using the filtered back projection (FBP), IR, and two types of DLR (AiCE Body and AiCE Body Sharp) algorithms. We set a square region of interest (ROI) on CT images of standard- and obese-sized low-contrast phantoms, slid different-sized moving average filters in the ROI vertically and horizontally in steps of 1 pixel, and calculated the standard deviation (SD) of the mean CT values for each filter size. The SD of the mean CT values was fitted with a curve inversely proportional to the filter size, and an apparent noise index was determined from the curve-fitting formula. The apparent noise index of AiCE Body Sharp images for a given mAs value was approximately 58, 23, and 18% lower than that of the FBP, AIDR 3D, and AiCE Body images, respectively. The apparent noise index was considered to reflect noise power spectrum values at lower spatial frequency. Moreover, the apparent noise index was inversely proportional to the square roots of the mAs values. Thus, the apparent noise index could be a useful indicator to quantify and compare texture noise on CT images obtained with different scan parameters and reconstruction algorithms.

The regression detectability index RDI for mammography images of breast phantoms with calcification-like objects and anatomical background

Currently, quality assurance measurements in mammography are performed on unprocessed images. For diagnosis, however, radiologists are provided with processed images. This image processing is optimised for images of human anatomy and therefore does not always perform satisfactorily with technical phantoms. To overcome this problem, it may be possible to use anthropomorphic phantoms reflecting the anatomic structure of the human breast in place of technical phantoms when carrying out task-specific quality assessment using model observers. However, the use of model observers is hampered by the fact that a large number of images needs to be acquired. A recently published novel observer called the regression detectability index (RDI) needs significantly fewer images, but requires the background of the images to be flat. Therefore, to be able to apply the RDI to images of anthropomorphic phantoms, the anatomic background needs to be removed. For this, a procedure in which the anatomical structures are fitted by thin plate spline (TPS) interpolation has been developed. When the object to be detected is small, such as a calcification-like lesion, it is shown that the anatomic background can be removed successfully by subtracting the TPS interpolation, which makes the background-free image accessible to the RDI. We have compared the detectability obtained by the RDI with TPS background subtraction to results of the channelized Hotelling observer (CHO) and human observers. With the RDI, results for the detectabilityd'can be obtained using 75% fewer images compared to the CHO, while the same uncertainty ofd'is achieved. Furthermore, the correlation ofd'(RDI) with the results of human observers is at least as good as that ofd'(CHO) with human observers.

Objective Task-Based Evaluation of Artificial Intelligence-Based Medical Imaging Methods:: Framework, Strategies, and Role of the Physician

Artificial intelligence-based methods are showing promise in medical imaging applications. There is substantial interest in clinical translation of these methods, requiring that they be evaluated rigorously. We lay out a framework for objective task-based evaluation of artificial intelligence methods. We provide a list of available tools to conduct this evaluation. We outline the important role of physicians in conducting these evaluation studies. The examples in this article are proposed in the context of PET scans with a focus on evaluating neural network-based methods. However, the framework is also applicable to evaluate other medical imaging modalities and other types of artificial intelligence methods.

Comparison of low-contrast detectability between uniform and anatomically realistic phantoms-influences on CT image quality assessment

Objectives

To evaluate the effects of anatomical phantom structure on task-based image quality assessment compared with a uniform phantom background.

Methods

Two neck phantom types of identical shape were investigated: a uniform type containing 10-mm lesions with 4, 9, 18, 30, and 38 HU contrast to the surrounding area and an anatomically realistic type containing lesions of the same size and location with 10, 18, 30, and 38 HU contrast. Phantom images were acquired at two dose levels (CTDIvol of 1.4 and 5.6 mGy) and reconstructed using filtered back projection (FBP) and adaptive iterative dose reduction 3D (AIDR 3D). Detection accuracy was evaluated by seven radiologists in a 4-alternative forced choice experiment.

Results

Anatomical phantom structure impaired lesion detection at all lesion contrasts (p < 0.01). Detectability in the anatomical phantom at 30 HU contrast was similar to 9 HU contrast in uniform images (91.1% vs. 89.5%). Detection accuracy decreased from 83.6% at 5.6 mGy to 55.4% at 1.4 mGy in uniform FBP images (p < 0.001), whereas AIDR 3D preserved detectability at 1.4 mGy (80.7% vs. 85% at 5.6 mGy, p = 0.375) and was superior to FBP (p < 0.001). In the assessment of anatomical images, superiority of AIDR 3D was not confirmed and dose reduction moderately affected detectability (74.6% vs. 68.2%, p = 0.027 for FBP and 81.1% vs. 73%, p = 0.018 for AIDR 3D).

Conclusions

A lesion contrast increase from 9 to 30 HU is necessary for similar detectability in anatomical and uniform neck phantom images. Anatomical phantom structure influences task-based assessment of iterative reconstruction and dose effects.

Key Points

• A lesion contrast increase from 9 to 30 HU is necessary for similar low-contrast detectability in anatomical and uniform neck phantom images.

• Phantom background structure influences task-based assessment of iterative reconstruction and dose effects.

• Transferability of CT assessment to clinical imaging can be expected to improve as the realism of the test environment increases.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-08248-3.

Assessment of task-based image quality for abdominal CT protocols linked with national diagnostic reference levels

Objectives

To assess task-based image quality for two abdominal protocols on various CT scanners. To establish a relationship between diagnostic reference levels (DRLs) and task-based image quality.

Methods

A protocol for the detection of focal liver lesions was used to scan an anthropomorphic abdominal phantom containing 8- and 5-mm low-contrast (20 HU) spheres at five CTDI_vol levels (4, 8, 12, 16, and 20 mGy) on 12 CTs. Another phantom with high-contrast calcium targets (200 HU) was scanned at 2, 4, 6, 10, and 15 mGy using a renal stones protocol on the same CTs. To assess the detectability, a channelized Hotelling observer was used for low-contrast targets and a non-prewhitening observer with an eye filter was used for high contrast targets. The area under the ROC curve and signal to noise ratio were used as figures of merit.

Results

For the detection of 8-mm spheres, the image quality reached a high level (mean AUC over all CTs higher than 0.95) at 11 mGy. For the detection of 5-mm spheres, the AUC never reached a high level of image quality. Variability between CTs was found, especially at low dose levels. For the search of renal stones, the AUC was nearly maximal even for the lowest dose level.

Conclusions

Comparable task-based image quality cannot be reached at the same dose level on all CT scanners. This variability implies the need for scanner-specific dose optimization.

Key Points

• There is an image quality variability for subtle low-contrast lesion detection in the clinically used dose range.

• Diagnostic reference levels were linked with task-based image quality metrics.

• There is a need for specific dose optimization for each CT scanner and clinical protocol.

The dosimetric impact of deep learning-based auto-segmentation of organs at risk on nasopharyngeal and rectal cancer

Purpose

To investigate the dosimetric impact of deep learning-based auto-segmentation of organs at risk (OARs) on nasopharyngeal and rectal cancer.

Methods and materials

Twenty patients, including ten nasopharyngeal carcinoma (NPC) patients and ten rectal cancer patients, who received radiotherapy in our department were enrolled in this study. Two deep learning-based auto-segmentation systems, including an in-house developed system (FD) and a commercial product (UIH), were used to generate two auto-segmented OARs sets (OAR_FD and OAR_UIH). Treatment plans based on auto-segmented OARs and following our clinical requirements were generated for each patient on each OARs sets (Plan_FD and Plan_UIH). Geometric metrics (Hausdorff distance (HD), mean distance to agreement (MDA), the Dice similarity coefficient (DICE) and the Jaccard index) were calculated for geometric evaluation. The dosimetric impact was evaluated by comparing Plan_FD and Plan_UIH to original clinically approved plans (Plan_Manual) with dose-volume metrics and 3D gamma analysis. Spearman’s correlation analysis was performed to investigate the correlation between dosimetric difference and geometric metrics.

Results

FD and UIH could provide similar geometric performance in parotids, temporal lobes, lens, and eyes (DICE, p > 0.05). OAR_FD had better geometric performance in the optic nerves, oral cavity, larynx, and femoral heads (DICE, p < 0.05). OAR_UIH had better geometric performance in the bladder (DICE, p < 0.05). In dosimetric analysis, both Plan_FD and Plan_UIH had nonsignificant dosimetric differences compared to Plan_Manual for most PTV and OARs dose-volume metrics. The only significant dosimetric difference was the max dose of the left temporal lobe for Plan_FD vs. Plan_Manual (p = 0.05). Only one significant correlation was found between the mean dose of the femoral head and its HD index (R = 0.4, p = 0.01), there is no OARs showed strong correlation between its dosimetric difference and all of four geometric metrics.

Conclusions

Deep learning-based OARs auto-segmentation for NPC and rectal cancer has a nonsignificant impact on most PTV and OARs dose-volume metrics. Correlations between the auto-segmentation geometric metric and dosimetric difference were not observed for most OARs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-021-01837-y.

Polyacrylamide hydrogel phantoms for performance evaluation of multispectral photoacoustic imaging systems

As photoacoustic imaging (PAI) begins to mature and undergo clinical translation, there is a need for well-validated, standardized performance test methods to support device development, quality control, and regulatory evaluation. Despite recent progress, current PAI phantoms may not adequately replicate tissue light and sound transport over the full range of optical wavelengths and acoustic frequencies employed by reported PAI devices. Here we introduce polyacrylamide (PAA) hydrogel as a candidate material for fabricating stable phantoms with well-characterized optical and acoustic properties that are biologically relevant over a broad range of system design parameters. We evaluated suitability of PAA phantoms for conducting image quality assessment of three PAI systems with substantially different operating parameters including two commercial systems and a custom system. Imaging results indicated that appropriately tuned PAA phantoms are useful tools for assessing and comparing PAI system image quality. These phantoms may also facilitate future standardization of performance test methodology.

DeepAMO: a multi-slice, multi-view anthropomorphic model observer for visual detection tasks performed on volume images

Purpose: We propose a deep learning-based anthropomorphic model observer (DeepAMO) for image quality evaluation of multi-orientation, multi-slice image sets with respect to a clinically realistic 3D defect detection task. Approach: The DeepAMO is developed based on a hypothetical model of the decision process of a human reader performing a detection task using a 3D volume. The DeepAMO is comprised of three sequential stages: defect segmentation, defect confirmation (DC), and rating value inference. The input to the DeepAMO is a composite image, typical of that used to view 3D volumes in clinical practice. The output is a rating value designed to reproduce a human observer's defect detection performance. In stages 2 and 3, we propose: (1) a projection-based DC block that confirms defect presence in two 2D orthogonal orientations and (2) a calibration method that "learns" the mapping from the features of stage 2 to the distribution of observer ratings from the human observer rating data (thus modeling inter- or intraobserver variability) using a mixture density network. We implemented and evaluated the DeepAMO in the context of   Tc 99 m -DMSA SPECT imaging. A human observer study was conducted, with two medical imaging physics graduate students serving as observers. A 5 × 2 -fold cross-validation experiment was conducted to test the statistical equivalence in defect detection performance between the DeepAMO and the human observer. We also compared the performance of the DeepAMO to an unoptimized implementation of a scanning linear discriminant observer (SLDO). Results: The results show that the DeepAMO's and human observer's performances on unseen images were statistically equivalent with a margin of difference ( Δ AUC ) of 0.0426 at p < 0.05 , using 288 training images. A limited implementation of an SLDO had a substantially higher AUC (0.99) compared to the DeepAMO and human observer. Conclusion: The results show that the DeepAMO has the potential to reproduce the absolute performance, and not just the relative ranking of human observers on a clinically realistic defect detection task, and that building conceptual components of the human reading process into deep learning-based models can allow training of these models in settings where limited training images are available.

Patient-based Performance Assessment for Pediatric Abdominal CT: An Automated Monitoring System Based on Lesion Detectability and Radiation Dose

RATIONALE AND OBJECTIVE

To deploy an automated tool for evaluating pediatric body computed tomography (CT) performance utilizing metrics of radiation dose and image quality for the task of liver lesion detection.

MATERIALS AND METHODS

This IRB approved retrospective investigation used 507 IV-contrast-enhanced abdominopelvic CT scans of pediatric patients (<18 years) between June 2014 and November 2017 acquired on three scanner models from two manufacturers. The scans were evaluated in terms of radiation metrics (CTDI_vol, DLP, and SSDE) as well as task-based performance based on the clinical task of detecting a 5 mm liver lesion with a 10 HU attenuation difference from background liver. An informatics algorithm extracted a previously-validated quantitative detectability index (d') from each case reflective of the likelihood of detecting a liver lesion. The results were analyzed in terms of the relationship between d' and radiation dose metrics.

RESULTS

There was minimal SSDE variability by age. Median SSDE at 100 kV on one scanner model was 5.2 mGy (5.0-5.4 mGy interquartile range). However, when assessing image quality by applying d', the age groups separated such that the younger patients had higher d' values than older patients. Similar trends were seen in all scanners.

CONCLUSIONS

An automated method to assess clinical image quality for pediatric CT provided a metric of image quality that varied as expected across ages (i.e., higher quality for younger patients). This tool affords the establishment of a quality reference level that, in addition to dose estimations currently available, would allow for enhanced assessment (e.g., facilitated audit) of CT imaging performance.

A physically cryptographic hoteling observer for nuclear warhead verification

When verifying the authenticity for nuclear warheads dismantlement toward a treaty partner's obligation, nuclear arms verification technologies are critical since only nuclear disarmament treaties are not sufficient to neutralize the existential threat of nuclear weapons. In this work, we present a verification method combining a numerical observer model and physical encryption techniques. The performance of the method is quantified by Monte Carlo simulations with several typical deception scenarios. Simulation results show this method can efficiently complete identification tasks in the presence of noise (<5 %) and source-term variability, meanwhile exhibiting high security against brute-force attacks which reconstruct detection data by the exhaustive method.

Task-based assessment of neck CT protocols using patient-mimicking phantoms-effects of protocol parameters on dose and diagnostic performance

Objectives

To assess how modifying multiple protocol parameters affects the dose and diagnostic performance of a neck CT protocol using patient-mimicking phantoms and task-based methods.

Methods

Six patient-mimicking neck phantoms containing hypodense lesions of 1 cm diameter and 30 HU contrast and one non-lesion phantom were examined with 36 CT protocols. All possible combinations of the following parameters were investigated: 100- and 120-kVp tube voltage; tube current modulation (TCM) noise levels of SD 7.5, 10, and 14; pitches of 0.637, 0.813, and 1.388; filtered back projection (FBP); and iterative reconstruction (AIDR 3D). Dose-length products (DLPs) and lesion detectability (assessed by 14 radiologists) were compared with the clinical standard protocol (120 kVp, TCM SD 7.5, 0.813 pitch, AIDR 3D).

Results

The DLP of the standard protocol was 25 mGy•cm; the area under the curve (AUC) was 0.839 (95%CI: 0.790–0.888). Combined effects of tube voltage reduction to 100 kVp and TCM noise level increase to SD 10 optimized protocol performance by improving dose (7.3 mGy•cm) and detectability (AUC 0.884, 95%CI: 0.844–0.924). Diagnostic performance was significantly affected by the TCM noise level at 120 kVp (AUC 0.821 at TCM SD 7.5 vs. 0.776 at TCM SD 14, p = 0.003), but not at 100-kVp tube voltage (AUC 0.839 at TCM SD 7.5 vs. 0.819 at TCM SD 14, p = 0.354), the reconstruction method at 100 kVp (AUC 0.854 for AIDR 3D vs. 0.806 for FBP, p < 0.001), but not at 120-kVp tube voltage (AUC 0.795 for AIDR 3D vs. 0.793 for FBP, p = 0.822), and the tube voltage for AIDR 3D reconstruction (p < 0.001), but not for FBP (p = 0.226).

Conclusions

Combined effects of 100-kVp tube voltage, TCM noise level of SD 10, a pitch of 0.813, and AIDR 3D resulted in an optimal neck protocol in terms of dose and diagnostic performance. Protocol parameters were subject to complex interactions, which created opportunities for protocol improvement.

Key Points

• A task-based approach using patient-mimicking phantoms was employed to optimize a CT system for neck imaging through systematic testing of protocol parameters.

• Combined effects of 100-kVp tube voltage, TCM noise level of SD 10, a pitch of 0.813, and AIDR 3D reconstruction resulted in an optimal protocol in terms of dose and diagnostic performance.

• Interactions of protocol parameters affect diagnostic performance and should be considered when optimizing CT techniques.

Electronic supplementary material

The online version of this article (10.1007/s00330-020-07374-8) contains supplementary material, which is available to authorized users.

Generating patient-specific virtual tumor populations with reaction-diffusion models and molecular imaging data

The use of mathematical tumor growth models coupled to noisy imaging data has been suggested as a possible component in the push towards precision medicine. We discuss the generation of population and patient-specific virtual populations in this context, providing in silico experiments to demonstrate how intra- and inter-patient heterogeneity can be estimated by applying rigorous statistical procedures to noisy molecular imaging data, and how the noise properties of such data can be analyzed to estimate uncertainties in predicted patient outcomes.

Evaluation of an integrated 3D-printed phantom for coronary CT angiography using iterative reconstruction algorithm

INTRODUCTION

3D-printed imaging phantoms are now increasingly available and used for computed tomography (CT) dose optimisation study and image quality analysis. The aim of this study was to evaluate the integrated 3D-printed cardiac insert phantom when evaluating iterative reconstruction (IR) algorithm in coronary CT angiography (CCTA) protocols.

METHODS

The 3D-printed cardiac insert phantom was positioned into a chest phantom and scanned with a 16-slice CT scanner. Acquisitions were performed with CCTA protocols using 120 kVp at four different tube currents, 300, 200, 100 and 50 mA (protocols A, B, C and D, respectively). The image data sets were reconstructed with a filtered back projection (FBP) and three different IR algorithm strengths. The image quality metrics of image noise, signal-noise ratio (SNR) and contrast-noise ratio (CNR) were calculated for each protocol.

RESULTS

Decrease in dose levels has significantly increased the image noise, compared to FBP of protocol A (P < 0.001). As a result, the SNR and CNR were significantly decreased (P < 0.001). For FBP, the highest noise with poor SNR and CNR was protocol D with 19.0 ± 1.6 HU, 18.9 ± 2.5 and 25.1 ± 3.6, respectively. For IR algorithm, the highest strength (AIDR3Dstrong ) yielded the lowest noise with excellent SNR and CNR.

CONCLUSIONS

The use of IR algorithm and increasing its strengths have reduced noise significantly and thus increased the SNR and CNR when compared to FBP. Therefore, this integrated 3D-printed phantom approach could be used for dose optimisation study and image quality analysis in CCTA protocols.

Development of a method to create uniform phantoms for task-based assessment of CT image quality

Purpose

To develop a customized method to produce uniform phantoms for task‐based assessment of CT image quality.

Methods

Contrasts between polymethyl methacrylate (PMMA) and fructose solutions of different concentrations (240, 250, 260, 280, 290, 300, 310, 320, 330, and 340 mg/mL) were calculated. A phantom was produced by laser cutting PMMA slabs to the shape of a patient’s neck. An opening of 10 mm diameter was cut into the left parapharyngeal space. An angioplasty balloon was inserted and filled with the fructose solutions to simulate low‐contrast lesions. The phantom was scanned with six tube currents. Images were reconstructed with filtered back projection (FBP) and adaptive iterative dose reduction 3D (AIDR 3D). Calculated and measured contrasts were compared. The phantom was evaluated in a detectability experiment using images with 4 and 20 HU lesion contrast.

Results

Low‐contrast lesions of 4, 9, 11, 13, 18, 20, 24, 30, 35, and 37 HU contrast were simulated. Calculated and measured contrasts correlated excellently (r = 0.998; 95% confidence interval: 0.991 to 1). The mean ± SD difference was 0.41 ± 2.32 HU (P < 0.0001). Detection accuracy and reader confidence were 62.9 ± 18.2% and 1.58 ± 0.68 for 4 HU lesion contrast and 99.6 ± 1.3% and 4.27 ± 0.92 for 20 HU lesion contrast (P < 0.0001), confirming that the method produced lesions at the threshold of detectability.

Conclusion

A cost‐effective and flexible approach was developed to create uniform phantoms with low‐contrast signals. The method should facilitate access to customized phantoms for task‐based image quality assessment.

RDI[Formula: see text]a regression detectability index for quality assurance in: x-ray imaging

Novel iterative image reconstruction methods can help reduce the required radiation dose in x-ray diagnostics such as computed tomography (CT), while maintaining sufficient image quality. Since some of the established image quality measures are not appropriate for reliably judging the quality of images derived by iterative methods, alternative approaches such as task-specific quality assessment would be highly desirable for acceptance or constancy testing. Task-based image quality methods are also closer to tasks performed by the radiologists, such as lesion detection. However, this approach is usually hampered by a huge workload, since hundreds of images are usually required for its application. It is demonstrated that the proposed approach works reliably on the basis of significantly fewer images, and that it correlates well with results obtained from human observers.

Frequency-dependent signal and noise in spectroscopic x-ray imaging

Purpose

We present a new framework for theoretical analysis of the noise power spectrum (NPS) of photon‐counting x‐ray detectors, including simple photon‐counting detectors (SPCDs) and spectroscopic x‐ray detectors (SXDs), the latter of which use multiple energy thresholds to discriminate photon energies.

Methods

We show that the NPS of SPCDs and SXDs, including spatio‐energetic noise correlations, is determined by the joint probability density function (PDF) of deposited photon energies, which describes the probability of recording two photons of two different energies in two different elements following a single‐photon interaction. We present an analytic expression for this joint PDF and calculate the presampling and digital NPS of CdTe SPCDs and SXDs. We calibrate our charge sharing model using the energy response of a cadmium zinc telluride (CZT) spectroscopic x‐ray detector and compare theoretical results with Monte Carlo simulations.

Results

Our analysis shows that charge sharing increases pixel signal‐to‐noise ratio (SNR), but degrades the zero‐frequency signal‐to‐noise performance of SPCDs and SXDs. In all cases considered, this degradation was greater than 10%. Comparing the presampling NPS with the sampled NPS showed that degradation in zero‐frequency performance is due to zero‐frequency noise aliasing induced by charge sharing.

Conclusions

Noise performance, including spatial and energy correlations between elements and energy bins, are described by the joint PDF of deposited energies which provides a method of determining the photon‐counting NPS, including noise‐aliasing effects and spatio‐energetic effects in spectral imaging. Our approach enables separating noise due to x‐ray interactions from that associated with sampling, consistent with cascaded systems analysis of energy‐integrating systems. Our methods can be incorporated into task‐based assessment of image quality for the design and optimization of spectroscopic x‐ray detectors.

3D printing of anatomically realistic phantoms with detection tasks to assess the diagnostic performance of CT images

Objectives

Detectability experiments performed to assess the diagnostic performance of computed tomography (CT) images should represent the clinical situation realistically. The purpose was to develop anatomically realistic phantoms with low-contrast lesions for detectability experiments.

Methods

Low-contrast lesions were digitally inserted into a neck CT image of a patient. The original and the manipulated CT images were used to create five phantoms: four phantoms with lesions of 10, 20, 30, and 40 HU contrast and one phantom without any lesion. Radiopaque 3D printing with potassium-iodide-doped ink (600 mg/mL) was used. The phantoms were scanned with different CT settings. Lesion contrast was analyzed using HU measurement. A 2-alternative forced choice experiment was performed with seven radiologists to study the impact of lesion contrast on detection accuracy and reader confidence (1 = lowest, 5 = highest).

Results

The phantoms reproduced patient size, shape, and anatomy. Mean ± SD contrast values of the low-contrast lesions were 9.7 ± 1.2, 18.2 ± 2, 30.2 ± 2.7, and 37.7 ± 3.1 HU for the 10, 20, 30, and 40 HU contrast lesions, respectively. Mean ± SD detection accuracy and confidence values were not significantly different for 10 and 20 HU lesion contrast (82.1 ± 6.3% vs. 83.9 ± 9.4%, p = 0.863 and 1.7 ± 0.4 vs. 1.8 ± 0.5, p = 0.159). They increased to 95 ± 5.7% and 2.6 ± 0.7 for 30 HU lesion contrast and 99.5 ± 0.9% and 3.8 ± 0.7 for 40 HU lesion contrast (p < 0.005).

Conclusions

A CT image was manipulated to produce anatomically realistic phantoms for low-contrast detectability experiments. The phantoms and our initial experiments provide a groundwork for the assessment of CT image quality in a clinical context.

Key Points

• Phantoms generated from manipulated CT images provide patient anatomy and can be used for detection tasks to evaluate the diagnostic performance of CT images.

• Radiologists are unconfident and unreliable in detecting hypodense lesions of 20 HU contrast and less in an anatomical neck background.

• Detectability experiments with anatomically realistic phantoms can assess CT image quality in a clinical context.

Electronic supplementary material

The online version of this article (10.1007/s00330-020-06808-7) contains supplementary material, which is available to authorized users.

Is there a role for image science in the brave new world of artificial intelligence?

Knowledge of the principles of image science is essential to the successful application of artificial intelligence in medical imaging.

Approximating the Ideal Observer and Hotelling Observer for Binary Signal Detection Tasks by Use of Supervised Learning Methods

It is widely accepted that the optimization of medical imaging system performance should be guided by task-based measures of image quality (IQ). Task-based measures of IQ quantify the ability of an observer to perform a specific task, such as detection or estimation of a signal (e.g., a tumor). For binary signal detection tasks, the Bayesian Ideal Observer (IO) sets an upper limit of observer performance and has been advocated for use in optimizing medical imaging systems and data-acquisition designs. Except in special cases, the determination of the IO test statistic is analytically intractable. Markov-chain Monte Carlo (MCMC) techniques can be employed to approximate the IO detection performance, but their reported applications have been limited to relatively simple object models. In cases where the IO test statistic is difficult to compute, the Hotelling Observer (HO) can be employed. To compute the HO test statistic, potentially large covariance matrices must be accurately estimated and subsequently inverted, which can present computational challenges. This paper investigates the supervised learning-based methodologies for approximating the IO and HO test statistics. Convolutional neural networks (CNNs) and single-layer neural networks (SLNNs) are employed to approximate the IO and HO test statistics, respectively. The numerical simulations were conducted for both signal-known-exactly (SKE) and signal-known-statistically (SKS) signal detection tasks. The considered background models include the lumpy object model and the clustered lumpy object model. The measurement noise models considered are Gaussian, Laplacian, and mixed Poisson-Gaussian. The performances of the supervised learning methods are assessed via receiver operating characteristic (ROC) analysis, and the results are compared to those produced by the use of traditional numerical methods or analytical calculations when feasible. The potential advantages of the proposed supervised learning approaches for approximating the IO and HO test statistics are discussed.

Validation of algorithmic CT image quality metrics with preferences of radiologists

PURPOSE

Automated assessment of perceptual image quality on clinical Computed Tomography (CT) data by computer algorithms has the potential to greatly facilitate data-driven monitoring and optimization of CT image acquisition protocols. The application of these techniques in clinical operation requires the knowledge of how the output of the computer algorithms corresponds to clinical expectations. This study addressed the need to validate algorithmic image quality measurements on clinical CT images with preferences of radiologists and determine the clinically acceptable range of algorithmic measurements for abdominal CT examinations.

MATERIALS AND METHODS

Algorithmic measurements of image quality metrics (organ HU, noise magnitude, and clarity) were performed on a clinical CT image dataset with supplemental measures of noise power spectrum from phantom images using techniques developed previously. The algorithmic measurements were compared to clinical expectations of image quality in an observer study with seven radiologists. Sets of CT liver images were selected from the dataset where images in the same set varied in terms of one metric at a time. These sets of images were shown via a web interface to one observer at a time. First, the observer rank ordered the CT images in a set according to his/her preference for the varying metric. The observer then selected his/her preferred acceptable range of the metric within the ranked images. The agreement between algorithmic and observer rankings of image quality were investigated and the clinically acceptable image quality in terms of algorithmic measurements were determined.

RESULTS

The overall rank-order agreements between algorithmic and observer assessments were 0.90, 0.98, and 1.00 for noise magnitude, liver parenchyma HU, and clarity, respectively. The results indicate a strong agreement between the algorithmic and observer assessments of image quality. Clinically acceptable thresholds (median) of algorithmic metric values were (17.8, 32.6) HU for noise magnitude, (92.1, 131.9) for liver parenchyma HU, and (0.47, 0.52) for clarity.

CONCLUSIONS

The observer study results indicated that these algorithms can robustly assess the perceptual quality of clinical CT images in an automated fashion. Clinically acceptable ranges of algorithmic measurements were determined. The correspondence of these image quality assessment algorithms to clinical expectations paves the way toward establishing diagnostic reference levels in terms of clinically acceptable perceptual image quality and data-driven optimization of CT image acquisition protocols.

Task-based image quality assessment in radiation therapy: initial characterization and demonstration with computer-simulation study

In the majority of current radiation therapy (RT) applications, image quality is still assessed subjectively or by utilizing physical measures. A novel theory that applies objective task-based image quality assessment in radiation therapy (IQA-in-RT) was recently proposed, in which the area under the therapeutic operating characteristic curve (AUTOC) was employed as the figure-of-merit (FOM) for evaluating RT effectiveness. Although theoretically more appealing than conventional subjective or physical measures, a comprehensive implementation and evaluation of this novel task-based IQA-in-RT theory is required for its further application in improving clinical RT. In this work, a practical and modular IQA-in-RT framework is presented for implementing this theory for the assessment of imaging components on the basis of RT treatment outcomes. Computer-simulation studies are conducted to demonstrate the feasibility and utility of the proposed IQA-in-RT framework in optimizing x-ray computed tomography (CT) pre-treatment imaging, including the optimization of CT imaging dose and image reconstruction parameters. The potential advantages of optimizing imaging components in the RT workflow by use of the AUTOC as the FOM are also compared against those of other physical measures. The results demonstrate that optimization using the AUTOC leads to selecting different parameters from those indicated by physical measures, potentially improving RT performance. The sources of systemic randomness and bias that affect the determination of the AUTOC are also analyzed. The presented work provides a practical solution for the further investigation and analysis of the task-based IQA-in-RT theory and advances its applications in improving RT clinical practice and cancer patient care.

Predicting Detection Performance on Security X-Ray Images as a Function of Image Quality

Developing methods to predict how image quality affects the task performance is a topic of great interest in many applications. While such studies have been performed in the medical imaging community, little work has been reported in the security X-ray imaging literature. In this paper, we develop models that predict the effect of image quality on the detection of the improvised explosive device components by bomb technicians in images taken using portable X-ray systems. Using a newly developed NIST-LIVE X-Ray Task Performance Database, we created a set of objective algorithms that predict bomb technician detection performance based on the measures of image quality. Our basic measures are traditional image quality indicators (IQIs) and perceptually relevant natural scene statistics (NSS)-based measures that have been extensively used in visible light image quality prediction algorithms. We show that these measures are able to quantify the perceptual severity of degradations and can predict the performance of expert bomb technicians in identifying threats. Combining NSS- and IQI-based measures yields even better task performance prediction than either of these methods independently. We also developed a new suite of statistical task prediction models that we refer to as quality inspectors of X-ray images (QUIX); we believe this is the first NSS-based model for security X-ray images. We also show that QUIX can be used to reliably predict conventional IQI metric values on the distorted X-ray images.

Determination of contrast-detail curves in mammography image quality assessment by a parametric model observer

A novel approach is proposed for the determination of contrast-detail curves in mammography image quality assessment. The approach is compared with current practice using virtual mammography. A binary parametric model observer is applied to images of the CDMAM phantom. The observer accounts for the simple disc shaped objects in the phantom and is applied separately to each cell of the phantom. For each of these applications, the area under the ROC curve (AUC) of the model observer is determined. The different AUCs, calculated from different applications of the parametric model observer, are then combined to a single contrast-detail curve quantifying the ability of the observer to detect details in the images. Virtual mammography is developed as a tool to simulate X-ray images of single CDMAM cells and to quantitatively assess the approach in comparison with current practice. It is shown that the proposed approach can lead to similar contrast-detail curves as current practice. The precision of the estimated contrast-detail curves is increased, i.e. using 5 images yields about the same precision for the proposed approach as 16 images when applying current practice. We conclude that contrast-detail curves in mammography image quality assessment can also be determined through the AUC of a binary parametric model observer. Since the proposed approach has higher precision than current practice, it is a promising candidate for contrast-detail analysis in mammography image quality assessment.

Technical Note: Design and implementation of a high-throughput pipeline for reconstruction and quantitative analysis of CT image data

PURPOSE

With recent substantial improvements in modern computing, interest in quantitative imaging with CT has seen a dramatic increase. As a result, the need to both create and analyze large, high-quality datasets of clinical studies has increased as well. At present, no efficient, widely available method exists to accomplish this. The purpose of this technical note is to describe an open-source high-throughput computational pipeline framework for the reconstruction and analysis of diagnostic CT imaging data to conduct large-scale quantitative imaging studies and to accelerate and improve quantitative imaging research.

METHODS

The pipeline consists of two, primary "blocks": reconstruction and analysis. Reconstruction is carried out via a graphics processing unit (GPU) queuing framework developed specifically for the pipeline that allows a dataset to be reconstructed using a variety of different parameter configurations such as slice thickness, reconstruction kernel, and simulated acquisition dose. The analysis portion then automatically analyzes the output of the reconstruction using "modules" that can be combined in various ways to conduct different experiments. Acceleration of analysis is achieved using cluster processing. Efficiency and performance of the pipeline are demonstrated using an example 142 subject lung screening cohort reconstructed 36 different ways and analyzed using quantitative emphysema scoring techniques.

RESULTS

The pipeline reconstructed and analyzed the 5112 reconstructed datasets in approximately 10 days, a roughly 72× speedup over previous efforts using the scanner for reconstructions. Tightly coupled pipeline quality assurance software ensured proper performance of analysis modules with regard to segmentation and emphysema scoring.

CONCLUSIONS

The pipeline greatly reduced the time from experiment conception to quantitative results. The modular design of the pipeline allows the high-throughput framework to be utilized for other future experiments into different quantitative imaging techniques. Future applications of the pipeline being explored are robustness testing of quantitative imaging metrics, data generation for deep learning, and use as a test platform for image-processing techniques to improve clinical quantitative imaging.

Evaluation of the Impact of Iterative Reconstruction Algorithms on Computed Tomography Texture Features of the Liver Parenchyma Using the Filtration-Histogram Method

OBJECTIVE

To evaluate whether computed tomography (CT) reconstruction algorithms affect the CT texture features of the liver parenchyma.

MATERIALS AND METHODS

This retrospective study comprised 58 patients (normal liver, n = 34; chronic liver disease [CLD], n = 24) who underwent liver CT scans using a single CT scanner. All CT images were reconstructed using filtered back projection (FBP), hybrid iterative reconstruction (IR) (iDOSE⁴), and model-based IR (IMR). On arterial phase (AP) and portal venous phase (PVP) CT imaging, quantitative texture analysis of the liver parenchyma using a single-slice region of interest was performed at the level of the hepatic hilum using a filtration-histogram statistic-based method with different filter values. Texture features were compared among the three reconstruction methods and between normal livers and those from CLD patients. Additionally, we evaluated the inter- and intra-observer reliability of the CT texture analysis by calculating intraclass correlation coefficients (ICCs).

RESULTS

IR techniques affect various CT texture features of the liver parenchyma. In particular, model-based IR frequently showed significant differences compared to FBP or hybrid IR on both AP and PVP CT imaging. Significant variation in entropy was observed between the three reconstruction algorithms on PVP imaging (p < 0.05). Comparison between normal livers and those from CLD patients revealed that AP images depend more strongly on the reconstruction method used than PVP images. For both inter- and intra-observer reliability, ICCs were acceptable (> 0.75) for CT imaging without filtration.

CONCLUSION

CT texture features of the liver parenchyma evaluated using the filtration-histogram method were significantly affected by the CT reconstruction algorithm used.

Model and human observer reproducibility for detection of microcalcification clusters in digital breast tomosynthesis images of three-dimensionally structured test object

We compare the reproducibility of the human observers and a channelized Hotelling observer (CHO), when reading digital breast tomosynthesis (DBT) images of a physical phantom containing a breast simulating structured background and calcification clusters at three dose levels. The phantom is scanned 217 times on a Siemens Inspiration DBT system. Volumes of interest, with and without the calcification targets, are extracted and the human observers' percentage of correct (PC) scores is evaluated using a four-alternative forced choice method. A two-layer CHO is developed using the human observer results. The first layer consists of a localizing CHO that identifies the most conspicuous calcifications using two Laguerre-Gauss channels. Then a CHO with eight Gabor channels estimates the PC score for the calcification cluster. Observer reproducibility is estimated by bootstrapping, and the standard deviation (SD) is used as a figure of merit. The CHO closely approximated the human observer results for all the three dose levels with a correlation of > 0.97 . For the larger calcification cluster sizes, both observers have similar reproducibility, whereas the CHO is more reproducible for the smaller calcifications, with a maximum of 5.5 SD against 13.1 SD for the human observers. The developed CHO is a good candidate for automated reading of the calcification clusters of the structured phantom, with better reproducibility than the human readers for small calcifications.

Systematic approach to a channelized Hotelling model observer implementation for a physical phantom containing mass-like lesions: Application to digital breast tomosynthesis

PURPOSE

to develop a channelized model observer (CHO) that matches human reader (HR) scoring of a physical phantom containing breast simulating structure and mass lesion-like targets for use in quality control of digital breast tomosynthesis (DBT) imaging systems.

METHODS

A total of 108 DBT scans of the phantom was acquired using a Siemens Inspiration DBT system. The detectability of mass-like targets was evaluated by human readers using a 4-alternative forced choice (4-AFC) method. The percentage correct (PC) values were then used as the benchmark for CHO tuning, again using a 4-AFC method. Three different channel functions were considered: Gabor, Laguerre-Gauss and Difference of Gaussian. With regard to the observer template, various methods for generating the expected signal were studied along with the influence of the number of training images used to form the covariance matrix for the observer template. Impact of bias in the training process on the observer template was evaluated next, as well as HR and CHO reproducibility.

RESULTS

HR performance was most closely matched by 8 Gabor channels with tuned phase, orientation and frequency, using an observer template generated from training image data. Just 24 DBT image stacks were required to give robust CHO performance with 0% bias, although a bias of up to 33% in the training images also gave acceptable performance. CHO and HR reproducibility were similar (on average 3.2 PC versus 3.4 PC).

CONCLUSIONS

The CHO algorithm developed matches human reader performance and is therefore a promising candidate for automated readout of phantom studies.

Task-Based Model Observer Assessment of A Partial Model-Based Iterative Reconstruction Algorithm in Thoracic Oncologic Multidetector CT

To investigate the impact of a partial model-based iterative reconstruction (ASiR-V) on image quality in thoracic oncologic multidetector computed tomography (MDCT), using human and mathematical model observers. Twenty cancer patients examined with regular-dose thoracic-abdominal-pelvic MDCT were retrospectively included. Thoracic images reconstructed using a sharp kernel and filtered back-projection (reference) or ASiR-V (0-100%, 20% increments; follow-up) were analysed by three thoracic radiologists. Advanced quantitative physical metrics, including detectability indexes of simulated 4-mm-diameter solid non-calcified nodules and ground-glass opacities, were computed at regular and reduced doses using a custom-designed phantom. All three radiologists preferred higher ASiR-V levels (best = 80%). Increasing ASiR-V substantially decreased noise magnitude, with slight changes in noise texture. For high-contrast objects, changing the ASiR-V level had no major effect on spatial resolution; whereas for lower-contrast objects, increasing ASiR-V substantially decreased spatial resolution, more markedly at reduced dose. For both high- and lower-contrast pulmonary lesions, detectability remained excellent, regardless of ASiR-V and dose levels, and increased significantly with increasing ASiR-V levels (all p < 0.001). While high ASiR-V levels (80%) are recommended to detect solid non-calcified nodules and ground-glass opacities in regular-dose thoracic oncologic MDCT, care must be taken because, for lower-contrast pulmonary lesions, high ASiR-V levels slightly change noise texture and substantially decrease spatial resolution, more markedly at reduced dose.