Comparison of visual grading and free-response ROC analyses for assessment of image-processing algorithms in digital mammography

Visual grading experiments and optimization in CBCT dental implantology imaging: preliminary application of integrated visual grading regression

This study uses a general formulation of integrated visual grading regression (IVGR) and applies it to cone beam computed tomography (CBCT) scan data related to anatomical landmarks for dental implantology. The aim was to assess and predict a minimum acceptable dose for diagnostic imaging and reporting. A skull phantom was imaged with a CBCT unit at various diagnostic exposures. Key anatomical landmarks within the images were independently reviewed by three trained observers. Each provided an overall image quality score. Statistical analysis was carried out to examine the acceptability of the images taken, using an IVGR analysis that was formulized as a three-stage protocol including defining an integrated score, development of an ordinal regression, and investigation of the possibility for dose reduction through estimated parameters. For a unit increase in the logarithm of radiation dose, the odds ratio that the integrated score for an image assessed by observers being rated in a higher category was 3.940 (95% confidence interval: 1.016-15.280). When assessed by the observers, the minimum dose required to achieve a 75% probability for an image to be classified as at least acceptable was 1346.91 mGy·cm² dose area product (DAP), a 31% reduction compared to the 1962 mGy·cm² DAP default dosage of the CBCT unit. The kappa values of the intra and inter-observer reliability indicated moderate agreements, while a discrepancy among observers was also identified because each, as expected, perceived visibility differently. The results of this work demonstrate the IVGR's predictive value of dose saving in the effort to reduce dose to patients while maintaining reportable diagnostic image quality.

Validation of a candidate instrument to assess image quality in digital mammography using ROC analysis

PURPOSE

To validate a candidate instrument, to be used by different professionals to assess image quality in digital mammography (DM), against detection performance results.

METHODS

A receiver operating characteristics (ROC) study was conducted to assess the detection performance in DM images with four different image quality levels due to different quality issues. Fourteen expert breast radiologists from five countries assessed a set of 80 DM cases, containing 60 lesions (40 cancers, 20 benign findings) and 20 normal cases. A visual grading analysis (VGA) study using a previously-described candidate instrument was conducted to evaluate a subset of 25 of the images used in the ROC study. Eight radiologists that had participated in the ROC study, and seven expert breast-imaging physicists, evaluated this subset. The VGA score (VGAS) and the ROC and visual grading characteristics (VGC) areas under the curve (AUCROC and AUCVGC) were compared.

RESULTS

No large differences in image quality among the four levels were detected by either ROC or VGA studies. However, the ranking of the four levels was consistent: level 1 (partial AUCROC: 0.070, VGAS: 6.77) performed better than levels 2 (0.066, 6.15), 3 (0.061, 5.82), and 4 (0.062, 5.37). Similarity between radiologists' and physicists' assessments was found (average VGAS difference of 10 %).

CONCLUSIONS

The results from the candidate instrument were found to correlate with those from ROC analysis, when used by either observer group. Therefore, it may be used by different professionals, such as radiologists, radiographers, and physicists, to assess clinically-relevant image quality variations in DM.

Development and content validity evaluation of a candidate instrument to assess image quality in digital mammography: A mixed-method study

PURPOSE

To develop a candidate instrument to assess image quality in digital mammography, by identifying clinically relevant features in images that are affected by lower image quality.

METHODS

Interviews with fifteen expert breast radiologists from five countries were conducted and analysed by using adapted directed content analysis. During these interviews, 45 mammographic cases, containing 44 lesions (30 cancers, 14 benign findings), and 5 normal cases, were shown with varying image quality. The interviews were performed to identify the structures from breast tissue and lesions relevant for image interpretation, and to investigate how image quality affected the visibility of those structures. The interview findings were used to develop tentative items, which were evaluated in terms of wording, understandability, and ambiguity with expert breast radiologists. The relevance of the tentative items was evaluated using the content validity index (CVI) and modified kappa index (k*).

RESULTS

Twelve content areas, representing the content of image quality in digital mammography, emerged from the interviews and were converted into 29 tentative items. Fourteen of these items demonstrated excellent CVI ≥ 0.78 (k* > 0.74), one showed good CVI < 0.78 (0.60 ≤ k* ≤ 0.74), while fourteen were of fair or poor CVI < 0.78 (k* ≤ 0.59). In total, nine items were deleted and five were revised or combined resulting in 18 items.

CONCLUSIONS

By following a mixed-method methodology, a candidate instrument was developed that may be used to characterise the clinically-relevant impact that image quality variations can have on digital mammography.

Determination of optimum exposure parameters for dentoalveolar structures of the jaws using the CB MercuRay system with cluster signal-to-noise analysis

OBJECTIVE

To determine the optimum cone beam computed tomography exposure parameters for specific diagnostic tasks.

METHODS

A Teflon phantom attached to a half-mandible in a large container was scanned in dental (D), implant (I), and panoramic (P) modes. An identical phantom in a small container was scanned in D mode. Both were scanned at 60, 80, 100, and 120 kV. We evaluated the image quality of five anatomical structures [dentinoenamel junction (1), lamina dura and periodontal ligament space (2), trabecular pattern (3), cortex-spongy bone junction (4), and pulp chamber and root canal (5)] and analyzed the diagnostic image quality with cluster signal-to-noise analysis. We then evaluated correlations between the two image qualities and calculated the threshold of acceptable diagnostic image quality. Optimum exposure parameters were determined from images with acceptable diagnostic image quality.

RESULTS

For the small container, the optimum exposure parameters were D mode, 80 kV for (1), (3), and (4) and D mode, 100 kV for (5). For the large container, they were D mode, 120 kV for (1), (3), and (5) and D mode, 100 kV for (4). I mode, 120 kV reached the acceptable level for (4). No images reached the acceptable level for (2).

CONCLUSIONS

No optimum exposure parameters were identified for the evaluation of the lamina dura and periodontal ligament space. D mode was sufficient for the other structures; however, the tube voltage required for each structure differed. Smaller patients required lower tube voltage. I mode, 120 kV may be used for larger lesions.

Influence of study design on digital pathology image quality evaluation: the need to define a clinical task

Despite the current rapid advance in technologies for whole slide imaging, there is still no scientific consensus on the recommended methodology for image quality assessment of digital pathology slides. For medical images in general, it has been recommended to assess image quality in terms of doctors' success rates in performing a specific clinical task while using the images (clinical image quality, cIQ). However, digital pathology is a new modality, and already identifying the appropriate task is difficult. In an alternative common approach, humans are asked to do a simpler task such as rating overall image quality (perceived image quality, pIQ), but that involves the risk of nonclinically relevant findings due to an unknown relationship between the pIQ and cIQ. In this study, we explored three different experimental protocols: (1) conducting a clinical task (detecting inclusion bodies), (2) rating image similarity and preference, and (3) rating the overall image quality. Additionally, within protocol 1, overall quality ratings were also collected (task-aware pIQ). The experiments were done by diagnostic veterinary pathologists in the context of evaluating the quality of hematoxylin and eosin-stained digital pathology slides of animal tissue samples under several common image alterations: additive noise, blurring, change in gamma, change in color saturation, and JPG compression. While the size of our experiments was small and prevents drawing strong conclusions, the results suggest the need to define a clinical task. Importantly, the pIQ data collected under protocols 2 and 3 did not always rank the image alterations the same as their cIQ from protocol 1, warning against using conventional pIQ to predict cIQ. At the same time, there was a correlation between the cIQ and task-aware pIQ ratings from protocol 1, suggesting that the clinical experiment context (set by specifying the clinical task) may affect human visual attention and bring focus to their criteria of image quality. Further research is needed to assess whether and for which purposes (e.g., preclinical testing) task-aware pIQ ratings could substitute cIQ for a given clinical task.

Precision imaging-its impact on image quality and diagnostic confidence in breast ultrasound examinations

OBJECTIVE

To determine the effect of noise-reducing innovation-precision imaging (PI)-on image quality and diagnostic efficacy in breast ultrasound.

METHODS

The study, which assessed four levels of PI from zero to three, consisted of two parts: image quality assessment and diagnostic efficacy evaluation. For the first part, 247 sets of ultrasound images displayed at each PI level were evaluated by 6 experienced breast imaging observers, by rating image quality using visual grading analysis on a 1-4 scale. For the diagnostic efficacy part 51 breast lesions were displayed at each PI level and scored 1-6 to generate a receiver operating characteristic (ROC) curve. These images were evaluated by radiologists and sonographers. Analyses were performed using non-parametric Friedman and Wilcoxon signed rank tests and a multireader multicase methodology.

RESULTS

Statistically, higher scores of image quality were observed with increased levels of PI than with the zero setting (p < 0.001). The ROC analysis did not demonstrate any significant change in diagnostic efficacy, with mean scores for all observers being 0.79, 0.80, 0.81 and 0.81 for settings zero, one, two and three, respectively.

CONCLUSION

This study suggested a perceived improvement in image quality with increasing levels of PI; however, no changes in diagnostic efficacy were noted. The importance of looking at the impact of new imaging technologies in a multifaceted way is emphasized.

ADVANCES IN KNOWLEDGE

To our knowledge, this is the first article investigating the impact of the PI algorithm on ultrasound image quality and breast lesion characterization.