A method to analyse observer disagreement in visual grading studies: example of assessed image quality in paediatric cerebral multidetector CT images

Quality assessment of anterior segment OCT images: Development and validation of quality criteria

BACKGROUND

The utility of medical imaging is dependant on image quality. We aimed to develop and validate quality criteria for ocular anterior segment optical coherence tomography (AS-OCT) images.

METHODS

We undertook a cross-sectional study using AS-OCT images from patients aged 6-16. A novel three-level grading system (good, limited or poor) was developed based on the presence of image artefact (categorised as lid, eyelash, cropping, glare, or movement artefact). Three independent experts graded 2825 images, with agreement assessed using confusion matrices and intraclass correlation coefficients (ICC) for each parameter.

RESULTS

There was very good inter-grader IQA agreement assessing image quality with ICC 0.85 (95 %CI: 0.84-0.87). The most commonly occurring artefact was eyelash artefact (1008/2825 images, 36 %). Graders labelled 621/2825 (22 %) images as good and 384 (14 %) as poor. There was complete agreement at either end of the confusion matrix with no 'good' images labelled as 'poor' by other graders, and vice versa. Similarly, there was very good agreement when assessing presence of lash (0.96,0.94-0.98), movement (0.97,0.96-0.99), glare (0.82,0.80-0.84) and cropping (0.90,0.88-0.92).

CONCLUSIONS

The novel image quality assessment criteria (IQAC) described here have good interobserver agreement overall, and excellent agreement on the differentiation between 'good' and 'poor' quality images. The large proportion of images graded as 'limited' suggests the need for refine this classification, using the specific IQAC features, for which we also report high interobserver agreement. These findings support the future potential for wider clinical and community care implementation of AS-OCT for the diagnosis and monitoring of ocular disease.

Stapes visualization by ultra-high resolution CT in cadaveric heads: A preliminary study

PURPOSE

This study aimed to assess stapes visualization using an ultra-high resolution computed tomography (U-HRCT).

METHOD

Sixty ears from 30 cadaveric human heads were scanned by both U-HRCT and 128-section multislice CT (MSCT) with clinical parameters. Image quality of the stapes head, anterior and posterior crura, footplate, incudostapedial joint and stapedial muscle within the pyramidal eminence was scored using a 3-point Likert scale. Linear measurements of the stapes configuration were performed on U-HRCT.

RESULTS

The interobserver agreement for image qualitative score on U-HRCT was good to excellent (interobserver agreement coefficients 0.65-0.86). With the exception of the stapes head, U-HRCT achieved significantly higher qualitative scores than MSCT across all anatomical structures (Ps < 0.05). The total height of the stapes was measured to be 3.48 ± 0.33 mm. The height and width of the obturator foramen were 1.77 ± 0.28 mm and 2.19 ± 0.33 mm, respectively. The widths of the anterior and posterior crura were 0.20 ± 0.06 mm and 0.22 ± 0.06 mm, respectively. The thickness of the footplate was 0.22 ± 0.06 mm, and the angle of the incudostapedial joint was 95.91 ± 10.69°.

CONCLUSIONS

U-HRCT is capable of delineating fine structures of the stapes and provides linear data on dimensions of the stapes, which could be helpful for detecting stapes disease and making individualized surgical plans in the clinical setting.

Evaluating medical images using deep convolutional neural networks: A simulated CT phantom image study

BACKGROUND

Applied research on artificial intelligence, mainly in deep learning, is widely performed. If medical images can be evaluated using artificial intelligence, this could substantially improve examination efficiency.

OBJECTIVE

We investigated an evaluation system for medical images with different noise characteristics using a deep convolutional neural network.

METHODS

Simulated computed tomography images are the targets of the system. We used an AlexNet trained with natural images for the deep convolutional neural network and a support vector machine for classification. Synthetic computed tomography images with circular and rectangular signal bodies at different levels of contrast and added Gaussian noise were used for training and testing.

RESULTS

Two transfer learning methods were tested: classification by a re-trained support vector machine using the AlexNet features, and a method that fine-tuned the deep convolutional neural network. Using the first method, all the test image noise levels could be classified correctly. The fine-tuning method achieved an accuracy rate of 92.6%.

CONCLUSIONS

An image quality evaluation method using artificial intelligence will be useful for clinical images and different image quality indices in the future.

Image quality in coronary computed tomography angiography: influence of adaptive statistical iterative reconstruction at various radiation dose levels

Background Computed tomography (CT) technology is rapidly evolving and software solution developed to optimize image quality and/or lower radiation dose. Purpose To investigate the influence of adaptive statistical iterative reconstruction (ASIR) at different radiation doses in coronary CT angiography (CCTA) in detailed image quality. Material and Methods A total of 160 CCTA were reconstructed as follows: 55 scans with filtered back projection (FBP) (650 mA), 51 scans (455 mA) with 30% ASIR (ASIR30), and 54 scans (295 mA) with 60% ASIR (ASIR60). For each reconstruction, subjective image quality was assessed by five independent certified cardiologists using a visual grading analysis (VGA) with five predefined image quality criteria consisting of a 5-point scale. Objective measures were contrast, noise, and contrast-to-noise ratio (CNR). Results The CTDI_vol resulted in 10.3 mGy, 7.4 mGy, and 4.6 mGy for FBP, ASIR30, and ASIR60, respectively. Homogeneity of the left ventricular lumen was the sole aspect in which reconstruction algorithms differed with a decreasing effect for ASIR60 compared to FBP (estimated odds ratio [OR] = 0.49 [95% confidence interval (CI) = 0.32-0.76; P = 0.001]). Decreased sharpness and spatial- and low-contrast resolutions were observed when using ASIR instead of FBP, but differences were not statistically significant. Concerning objective measurements, noise increased significantly for ASIR30 (OR = 1.08; 95% CI = 1.02-1.14; P = 0.006) and ASIR60 (OR = 1.06; 95% CI = 1.01-1.12; P = 0.034) compared to FBP. Conclusion ASIR significantly decreased the subjectively assessed homogeneity of the left ventricular lumen and increased the objectively measured noise compared to FBP. Considering these results, ASIR at a reduced radiation dose should be implemented with caution.

Superior outcomes following cervical fusion vs. multimodal rehabilitation in a subgroup of randomized Whiplash-Associated-Disorders (WAD) patients indicating somatic pain origin-Comparison of outcome assessments made by four examiners from different disciplines

BACKGROUND AND AIMS

Whiplash-Associated Disorders (WAD) are characterized by great variability in long-term symptoms. Patients with central neck and movement-induced stabbing pain participated in a randomized study comparing cervical fusion and multimodal rehabilitation. As reported in our previous paper, more patients treated by cervical fusion than by rehabilitation experienced pain relief. Although patient reported outcome measures are a core component of outcome evaluation, independent examiner has been recommended. Because of the heterogeneity of WAD complaints the patients in our study were examined at baseline and follow-up by four experts representing neurology, orthopedics, psychology and physical medicine. The aim was to compare the professional assessments of change both regarding the possible impact of the different examiners' perspectives on individual patient's outcome, and also on the analysis of possible outcome differences between the treatment groups.

METHODS

WAD patients with long-term neck pain as the predominant symptom after a traffic accident were eligible. The neck pain origin should be in the midline and perceived as dull and aching, with sudden movement inducing midline stabbing pain. Of the 1,052 patients in contact with our team, 49 were eligible. The overall treatment effect was evaluated on a global outcome transitional scale. The criteria for the scale categories were defined by each expert's professional perspective on change in the whiplash complaints. Statistical methods that take account of the non-metric properties of ordered categorical data were used. Observed inter-expert disagreement was evaluated by the Svensson method that identifies and measures systematic group-related disagreement separately from disagreement caused by individual variation. Possible differences in the distributions of assessments on the expert-specific outcome scales between the treatment groups were analyzed by the Kruskal-Wallis test.

RESULTS

The per-protocol evaluation showed that a majority of the 18 patients who underwent fusion surgery were assessed as somewhat or much better, ranging from 67% to 78% depending on the expert. Corresponding proportions of improvement in the 17 patients treated by multimodal rehabilitation ranged from 29% to 53%. The statistical analyses confirmed better outcomes in the patients treated by fusion surgery, with p-values ranging from 0.003 to 0.04. The experts' assessments of intra-patient change disagreed more or less for all patients. The analyses of the paired comparisons confirmed that these disagreements could most probably be explained by the different profession-specific operational definitions of the outcome scales rather than by individual variations in data.

CONCLUSIONS

The multi-dimensional complexity of WAD-related complaints was comprehensively demonstrated by the inter-disciplinary disagreements in assessing intra-patient outcomes. The superiority of positive treatment effects in patients who underwent cervical fusion compared with multimodal rehabilitation was evident to all experts.

IMPLICATIONS

The results strengthen our previous opinion that neck pain in this subgroup of WAD patients has a somatic origin. More than one examiner is recommended for multi-dimensional outcome assessments.

Influence of adaptive statistical iterative reconstruction algorithm on image quality in coronary computed tomography angiography

Background

Coronary computed tomography angiography (CCTA) requires high spatial and temporal resolution, increased low contrast resolution for the assessment of coronary artery stenosis, plaque detection, and/or non-coronary pathology. Therefore, new reconstruction algorithms, particularly iterative reconstruction (IR) techniques, have been developed in an attempt to improve image quality with no cost in radiation exposure.

Purpose

To evaluate whether adaptive statistical iterative reconstruction (ASIR) enhances perceived image quality in CCTA compared to filtered back projection (FBP).

Material and Methods

Thirty patients underwent CCTA due to suspected coronary artery disease. Images were reconstructed using FBP, 30% ASIR, and 60% ASIR. Ninety image sets were evaluated by five observers using the subjective visual grading analysis (VGA) and assessed by proportional odds modeling. Objective quality assessment (contrast, noise, and the contrast-to-noise ratio [CNR]) was analyzed with linear mixed effects modeling on log-transformed data. The need for ethical approval was waived by the local ethics committee as the study only involved anonymously collected clinical data.

Results

VGA showed significant improvements in sharpness by comparing FBP with ASIR, resulting in odds ratios of 1.54 for 30% ASIR and 1.89 for 60% ASIR (P = 0.004). The objective measures showed significant differences between FBP and 60% ASIR (P < 0.0001) for noise, with an estimated ratio of 0.82, and for CNR, with an estimated ratio of 1.26.

Conclusion

ASIR improved the subjective image quality of parameter sharpness and, objectively, reduced noise and increased CNR.

A quality improvement project to reduce hypothermia in infants undergoing MRI scanning

BACKGROUND

Hypothermia prevention strategies during MRI scanning under general anesthesia in infants may pose a challenge due to the MRI scanner's technical constraints. Previous studies have demonstrated conflicting results related to increase or decrease in post-scan temperatures in children. We noted occurrences of post-scan hypothermia in anesthetized infants despite the use of routine passive warming techniques.

OBJECTIVE

The aims of our quality improvement project were (a) to identify variables associated with post-scan hypothermia in infants and (b) to develop and implement processes to reduce occurrence of hypothermia in neonatal intensive care unit (NICU) infants undergoing MRI.

MATERIALS AND METHODS

One hundred sixty-four infants undergoing MRI scanning were prospectively audited for post-scan body temperatures. A multidisciplinary team identified potential variables associated with post-scan hypothermia and designed preventative strategies: protocol development, risk factor identification, vigilance and use of a vacuum immobilizer. Another audit was performed, specifically focusing on NICU infants.

RESULTS

In the initial phase, we found that younger age (P = 0.002), lower weight (P = 0.005), lower pre-scan temperature (P < 0.01), primary anesthetic technique with propofol (P < 0.01), advanced airway devices (P = 0.02) and being in the NICU (P < 0.01) were associated with higher odds for developing post-scan decrease in body temperature. Quality improvement processes decreased the occurrence of hypothermia in NICU infants undergoing MRI scanning from 65% to 18% (95% confidence interval for the difference, 26-70%, P < 0.001).

CONCLUSION

Several variables, including being in the NICU, are associated with a decrease in post-scan temperature in infants undergoing MRI scanning under sedation/general anesthesia. Implementation of strategies to prevent hypothermia in infants may be challenging in the high-risk MRI environment. We were able to minimize this problem in clinical practice by applying quality improvement principles.

ASSESSMENT OF CLINICAL IMAGE QUALITY IN PAEDIATRIC ABDOMINAL CT EXAMINATIONS: DEPENDENCY ON THE LEVEL OF ADAPTIVE STATISTICAL ITERATIVE RECONSTRUCTION (ASiR) AND THE TYPE OF CONVOLUTION KERNEL

The purpose of this study was to investigate the effect of different combinations of convolution kernel and the level of Adaptive Statistical iterative Reconstruction (ASiR™) on diagnostic image quality as well as visualisation of anatomical structures in paediatric abdominal computed tomography (CT) examinations. Thirty-five paediatric patients with abdominal pain with non-specified pathology undergoing abdominal CT were included in the study. Transaxial stacks of 5-mm-thick images were retrospectively reconstructed at various ASiR levels, in combination with three convolution kernels. Four paediatric radiologists rated the diagnostic image quality and the delineation of six anatomical structures in a blinded randomised visual grading study. Image quality at a given ASiR level was found to be dependent on the kernel, and a more edge-enhancing kernel benefitted from a higher ASiR level. An ASiR level of 70 % together with the Soft™ or Standard™ kernel was suggested to be the optimal combination for paediatric abdominal CT examinations.

THE EFFECT OF ADAPTIVE STATISTICAL ITERATIVE RECONSTRUCTION ON THE ASSESSMENT OF DIAGNOSTIC IMAGE QUALITY AND VISUALISATION OF ANATOMICAL STRUCTURES IN PAEDIATRIC CEREBRAL CT EXAMINATIONS

The purpose of this study was to investigate the effect of adaptive statistical iterative reconstruction (ASiR) on the visualisation of anatomical structures and diagnostic image quality in paediatric cerebral computed tomography (CT) examinations. Forty paediatric patients undergoing routine cerebral CT were included in the study. The raw data from CT scans were reconstructed into stacks of 5 mm thick axial images at various levels of ASiR. Three paediatric radiologists rated six questions related to the visualisation of anatomical structures and one question on diagnostic image quality, in a blinded randomised visual grading study. The evaluated anatomical structures demonstrated enhanced visibility with increasing level of ASiR, apart from the cerebrospinal fluid space around the brain. In this study, 60 % ASiR was found to be the optimal level of ASiR for paediatric cerebral CT examinations. This shows that the commonly used 30 % ASiR may not always be the optimal level.

The impact of pediatric-specific dose modulation curves on radiation dose and image quality in head computed tomography

BACKGROUND

The volume of CT examinations has increased with resultant increases in collective dose values over the last decade.

OBJECTIVE

To analyze the impact of the tube current and voltage modulation for dose values and image quality of pediatric head CT examinations.

MATERIALS AND METHODS

Head CT examinations were performed on anthropomorphic phantoms and four pediatric age categories before and after the introduction of dedicated pediatric curves for tube voltage and current modulation. Local diagnostic reference levels were calculated. Visual grading characteristic image quality evaluation was performed by four pediatric neuroradiologists and image noise comparisons were performed.

RESULTS

Pediatric-specific modulation curves demonstrated a 49% decrease in mean radiation dose for phantom examinations. The local diagnostic reference levels (CTDIvol) for clinical examinations decreased by 52%, 41%, 46% and 40% for newborn, 5-, 10- and 15-year-old patients, respectively. Visual grading characteristic image quality was maintained for the majority of age categorizations (area under the curve = 0.5) and image noise measurements did not change (P = 0.693).

CONCLUSION

Pediatric-specific dose modulation curves resulted in an overall mean dose reduction of 45% with no significant differences in subjective or objective image quality findings.

Measures of agreement between many raters for ordinal classifications

Screening and diagnostic procedures often require a physician's subjective interpretation of a patient's test result using an ordered categorical scale to define the patient's disease severity. Because of wide variability observed between physicians' ratings, many large-scale studies have been conducted to quantify agreement between multiple experts' ordinal classifications in common diagnostic procedures such as mammography. However, very few statistical approaches are available to assess agreement in these large-scale settings. Many existing summary measures of agreement rely on extensions of Cohen's kappa. These are prone to prevalence and marginal distribution issues, become increasingly complex for more than three experts, or are not easily implemented. Here we propose a model-based approach to assess agreement in large-scale studies based upon a framework of ordinal generalized linear mixed models. A summary measure of agreement is proposed for multiple experts assessing the same sample of patients' test results according to an ordered categorical scale. This measure avoids some of the key flaws associated with Cohen's kappa and its extensions. Simulation studies are conducted to demonstrate the validity of the approach with comparison with commonly used agreement measures. The proposed methods are easily implemented using the software package R and are applied to two large-scale cancer agreement studies.

An evaluation of the use of oral contrast media in abdominopelvic CT

OBJECTIVES

To evaluate the diagnostic efficacy of different oral contrast media (OCM) for abdominopelvic CT examinations performed for follow-up general oncological indications. The objectives were to establish anatomical image quality criteria for abdominopelvic CT; use these criteria to evaluate and compare image quality using positive OCM, neutral OCM and no OCM; and evaluate possible benefits for the medical imaging department.

METHODS

Forty-six adult patients attending a follow-up abdominopelvic CT for general oncological indications and who had a previous abdominopelvic CT with positive OCM (n = 46) were recruited and prospectively placed into either the water (n = 25) or no OCM (n = 21) group. Three radiologists performed absolute visual grading analysis (VGA) to assess image quality by grading the fulfilment of 24 anatomical image quality criteria.

RESULTS

Visual grading characteristics (VGC) analysis of the data showed comparable image quality with regards to reproduction of abdominal structures, bowel discrimination, presence of artefacts, and visualization of the amount of intra-abdominal fat for the three OCM protocols.

CONCLUSION

All three OCM protocols provided similar image quality for follow-up abdominopelvic CT for general oncological indications.

KEY POINTS

• Positive oral contrast media are routinely used for abdominopelvic multidetector computed tomography • Experimental study comparing image quality using three different oral contrast materials • Three different oral contrast materials result in comparable CT image quality • Benefits for patients and medical imaging department.

Paediatric x-ray radiation dose reduction and image quality analysis

Collaboration of multiple staff groups has resulted in significant reduction in the risk of radiation-induced cancer from radiographic x-ray exposure during childhood. In this study at an acute NHS hospital trust, a preliminary audit identified initial exposure factors. These were compared with European and UK guidance, leading to the introduction of new factors that were in compliance with European guidance on x-ray tube potentials. Image quality was assessed using standard anatomical criteria scoring, and visual grading characteristics analysis assessed the impact on image quality of changes in exposure factors. This analysis determined the acceptability of gradual radiation dose reduction below the European and UK guidance levels. Chest and pelvis exposures were optimised, achieving dose reduction for each age group, with 7%-55% decrease in critical organ dose. Clinicians confirmed diagnostic image quality throughout the iterative process. Analysis of images acquired with preliminary and final exposure factors indicated an average visual grading analysis result of 0.5, demonstrating equivalent image quality. The optimisation process and final radiation doses are reported for Carestream computed radiography to aid other hospitals in minimising radiation risks to children.

Quantifying the potential for dose reduction with visual grading regression

Objectives To propose a method to study the effect of exposure settings on image quality and to estimate the potential for dose reduction when introducing dose-reducing measures. Methods Using the framework of visual grading regression (VGR), a log(mAs) term is included in the ordinal logistic regression equation, so that the effect of reducing the dose can be quantitatively related to the effect of adding post-processing. In the ordinal logistic regression, patient and observer identity are treated as random effects using generalised linear latent and mixed models. The potential dose reduction is then estimated from the regression coefficients. The method was applied in a single-image study of coronary CT angiography (CTA) to evaluate two-dimensional (2D) adaptive filters, and in an image-pair study of abdominal CT to evaluate 2D and three-dimensional (3D) adaptive filters. Results For five image quality criteria in coronary CTA, dose reductions of 16-26% were predicted when adding 2D filtering. Using five image quality criteria for abdominal CT, it was estimated that 2D filtering permits doses were reduced by 32-41%, and 3D filtering by 42-51%. Conclusions VGR including a log(mAs) term can be used for predictions of potential dose reduction that may be useful for guiding researchers in designing subsequent studies evaluating diagnostic value. With appropriate statistical analysis, it is possible to obtain direct numerical estimates of the dose-reducing potential of novel acquisition, reconstruction or post-processing techniques.

Quantifying the potential for dose reduction with visual grading regression

OBJECTIVES

To propose a method to study the effect of exposure settings on image quality and to estimate the potential for dose reduction when introducing dose-reducing measures.

METHODS

Using the framework of visual grading regression (VGR), a log(mAs) term is included in the ordinal logistic regression equation, so that the effect of reducing the dose can be quantitatively related to the effect of adding post-processing. In the ordinal logistic regression, patient and observer identity are treated as random effects using generalised linear latent and mixed models. The potential dose reduction is then estimated from the regression coefficients. The method was applied in a single-image study of coronary CT angiography (CTA) to evaluate two-dimensional (2D) adaptive filters, and in an image-pair study of abdominal CT to evaluate 2D and three-dimensional (3D) adaptive filters.

RESULTS

For five image quality criteria in coronary CTA, dose reductions of 16-26% were predicted when adding 2D filtering. Using five image quality criteria for abdominal CT, it was estimated that 2D filtering permits doses were reduced by 32-41%, and 3D filtering by 42-51%.

CONCLUSIONS

VGR including a log(mAs) term can be used for predictions of potential dose reduction that may be useful for guiding researchers in designing subsequent studies evaluating diagnostic value. With appropriate statistical analysis, it is possible to obtain direct numerical estimates of the dose-reducing potential of novel acquisition, reconstruction or post-processing techniques.

In Vivo Computed Tomography as a Research Tool to Investigate Asthma and COPD: Where Do We Stand?

Computed tomography (CT) is a clinical tool widely used to assess and followup asthma and chonic obstructive pulmonary disease (COPD) in humans. Strong efforts have been made the last decade to improve this technique as a quantitative research tool. Using semiautomatic softwares, quantification of airway wall thickness, lumen area, and bronchial wall density are available from large to intermediate conductive airways. Skeletonization of the bronchial tree can be built to assess its three-dimensional geometry. Lung parenchyma density can be analysed as a surrogate of small airway disease and emphysema. Since resident cells involve airway wall and lung parenchyma abnormalities, CT provides an accurate and reliable research tool to assess their role in vivo. This litterature review highlights the most recent advances made to assess asthma and COPD with CT, and also their drawbacks and the place of CT in clarifying the complex physiopathology of both diseases.