Reliability of Coronary Artery Calcium Severity Assessment on Non-Electrocardiogram-Gated CT: A Meta-Analysis

Coronary artery calcium scoring assessment in ultra-low-dose chest computed tomography

OBJECTIVES

To investigate the effect of non-electrocardiogram (ECG) -triggered ultra-low-dose CT (ULD-CT) with different reconstruction protocols on coronary artery calcium (CAC) scoring assessment, compared with ECG-triggered CAC CT (CAC-CT).

METHODS

This prospective study included 115 patients who underwent CAC-CT and ULD-CT scans under the same topogram images. CAC-CT adopted a prospective ECG-triggered sequential acquisition with a tube potential of 120 kV, and the reconstruction protocol was standard Qr36 + slice 3 mm (CACQr-3mm group). ULD-CT adopted a non-ECG-triggered high-pitch acquisition with a tube potential of Sn100 kV, and four groups of images (named ULDQr-3mm, ULDSa-3mm, ULDQr-1.5mm, and ULDSa-1.5mm) were reconstructed using different reconstruction algorithms (standard Qr36, kV-independent Sa36) and slice thicknesses (3 mm, 1.5 mm). The accuracy of CAC detection by ULD-CT was calculated. The agreement of the CAC score between ULD-CT and CAC-CT scans was assessed using intraclass correlation coefficients (ICC) and Bland-Altman plot, and the agreement of risk categorization was assessed using weighted kappa.

RESULTS

The sensitivity and specificity of the ULDSa-1.5mm group for detecting positive CAC were 100% and 97.4%, respectively (k = 0.980). The CAC score for the ULDSa-3mm and ULDSa-1.5mm groups demonstrated excellent agreement with the CACQr-3mm group (ICC = 0.992, 0.990, respectively), with a mean difference of -12.3 and - 12.4. The agreement of risk categorization based on absolute and percentile CAC score between the ULDSa-1.5mm and CACQr-3mm groups was excellent (weighted k = 0.954, 0.983, respectively), and risk reclassification rates were low (3.5%, 2.8%, respectively). The effective dose was reduced by approximately 77.2% for the ULD-CT compared to the CAC-CT (0.18 mSv vs. 0.79 mSv, p < 0.001).

CONCLUSION

Reconstruction with a 1.5-mm slice thickness and kV-independent iterative algorithmic protocol in ULD-CT yielded excellent agreement in CAC score quantification and risk categorization compared with ECG-triggered CAC-CT.

Fully automatic coronary calcium scoring in non-ECG-gated low-dose chest CT: comparison with ECG-gated cardiac CT

OBJECTIVES

To validate an artificial intelligence (AI)-based fully automatic coronary artery calcium (CAC) scoring system on non-electrocardiogram (ECG)-gated low-dose chest computed tomography (LDCT) using multi-institutional datasets with manual CAC scoring as the reference standard.

METHODS

This retrospective study included 452 subjects from three academic institutions, who underwent both ECG-gated calcium scoring computed tomography (CSCT) and LDCT scans. For all CSCT and LDCT scans, automatic CAC scoring (CAC_auto) was performed using AI-based software, and manual CAC scoring (CAC_man) was set as the reference standard. The reliability and agreement of CAC_auto was evaluated and compared with that of CAC_man using intraclass correlation coefficients (ICCs) and Bland-Altman plots. The reliability between CAC_auto and CAC_man for CAC severity categories was analyzed using weighted kappa (κ) statistics.

RESULTS

CAC_auto on CSCT and LDCT yielded a high ICC (0.998, 95% confidence interval (CI) 0.998-0.999 and 0.989, 95% CI 0.987-0.991, respectively) and a mean difference with 95% limits of agreement of 1.3 ± 37.1 and 0.8 ± 75.7, respectively. CAC_auto achieved excellent reliability for CAC severity (κ = 0.918-0.972) on CSCT and good to excellent but heterogenous reliability among datasets (κ = 0.748-0.924) on LDCT.

CONCLUSIONS

The application of an AI-based automatic CAC scoring software to LDCT shows good to excellent reliability in CAC score and CAC severity categorization in multi-institutional datasets; however, the reliability varies among institutions.

KEY POINTS

• AI-based automatic CAC scoring on LDCT shows excellent reliability with manual CAC scoring in multi-institutional datasets. • The reliability for CAC score-based severity categorization varies among datasets. • Automatic scoring for LDCT shows a higher false-positive rate than automatic scoring for CSCT, and most common causes of a false-positive are image noise and artifacts for both CSCT and LDCT.

A Modified Length-Based Grading Method for Assessing Coronary Artery Calcium Severity on Non-Electrocardiogram-Gated Chest Computed Tomography: A Multiple-Observer Study

OBJECTIVE

To validate a simplified ordinal scoring method, referred to as modified length-based grading, for assessing coronary artery calcium (CAC) severity on non-electrocardiogram (ECG)-gated chest computed tomography (CT).

MATERIALS AND METHODS

This retrospective study enrolled 120 patients (mean age ± standard deviation [SD], 63.1 ± 14.5 years; male, 64) who underwent both non-ECG-gated chest CT and ECG-gated cardiac CT between January 2011 and December 2021. Six radiologists independently assessed CAC severity on chest CT using two scoring methods (visual assessment and modified length-based grading) and categorized the results as none, mild, moderate, or severe. The CAC category on cardiac CT assessed using the Agatston score was used as the reference standard. Agreement among the six observers for CAC category classification was assessed using Fleiss kappa statistics. Agreement between CAC categories on chest CT obtained using either method and the Agatston score categories on cardiac CT was assessed using Cohen's kappa. The time taken to evaluate CAC grading was compared between the observers and two grading methods.

RESULTS

For differentiation of the four CAC categories, interobserver agreement was moderate for visual assessment (Fleiss kappa, 0.553 [95% confidence interval {CI}: 0.496-0.610]) and good for modified length-based grading (Fleiss kappa, 0.695 [95% CI: 0.636-0.754]). The modified length-based grading demonstrated better agreement with the reference standard categorization with cardiac CT than visual assessment (Cohen's kappa, 0.565 [95% CI: 0.511-0.619 for visual assessment vs. 0.695 [95% CI: 0.638-0.752] for modified length-based grading). The overall time for evaluating CAC grading was slightly shorter in visual assessment (mean ± SD, 41.8 ± 38.9 s) than in modified length-based grading (43.5 ± 33.2 s) (P < 0.001).

CONCLUSION

The modified length-based grading worked well for evaluating CAC on non-ECG-gated chest CT with better interobserver agreement and agreement with cardiac CT than visual assessment.

Validation of deep learning-based fully automated coronary artery calcium scoring using non-ECG-gated chest CT in patients with cancer

This study aimed to demonstrate clinical feasibility of deep learning (DL)-based fully automated coronary artery calcium (CAC) scoring software using non-electrocardiogram (ECG)-gated chest computed tomography (CT) from patients with cancer. Overall, 913 patients with colorectal or gastric cancer who underwent non-contrast-enhanced chest CT between 2013 and 2015 were included. Agatston scores obtained by manual segmentation of CAC on chest CT were used as reference. Reliability of automated CAC score acquisition was evaluated using intraclass correlation coefficients (ICCs). The agreement for cardiovascular disease (CVD) risk stratification was assessed with linearly weighted k statistics. ICCs between the manual and automated CAC scores were 0.992 (95% CI, 0.991 and 0.993, p<0.001) for total Agatston scores, 0.863 (95% CI, 0.844 and 0.880, p<0.001) for the left main, 0.964 (95% CI, 0.959 and 0.968, p<0.001) for the left anterior descending, 0.962 (95% CI, 0.956 and 0.966, p<0.001) for the left circumflex, and 0.980 (95% CI, 0.978 and 0.983, p<0.001) for the right coronary arteries. The agreement for cardiovascular risk was excellent (k=0.946, p<0.001). Current DL-based automated CAC software showed excellent reliability for Agatston score and CVD risk stratification using non-ECG gated CT scans and might allow the identification of high-risk cancer patients for CVD.

Progostic value of coronary artery calcium scores from 1.5 mm slice reconstructions of electrocardiogram-gated computed tomography scans in asymptomatic individuals

It is unknown whether the thinner slice reconstruction has added value relative to 3 mm reconstructions in predicting major adverse cardiac events (MACEs). This retrospective study included 550 asymptomatic individuals who underwent cardiac CT. Coronary artery calcium (CAC) scores and severity categories were assessed from 1.5 and 3 mm scans. CAC scores obtained from 1.5 and 3 mm scans were compared using Wilcoxon signed-rank tests. Cox proportional hazard models were developed to predict MACEs based on the degree of coronary artery stenosis on coronary CT angiography and the presence of CAC on both scans. Model performances were compared using the time-dependent ROC curve and integrated area under the curve (iAUC) methods. The CAC scores obtained from 1.5 mm scans were significantly higher than those from 3 mm scans (median, interquartile range 4.5[0–71] vs. 0[0–48.4]; p < 0.001). Models showed no difference in predictive accuracy of the presence of CAC between 1.5 and 3 mm scans (iAUC, 0.625 vs. 0.672). In conclusion, CAC scores obtained from 1.5 mm scans are significantly higher than those from 3 mm scans, but do not provide added prognostic value relative to 3 mm scans.

Quality Reporting of Systematic Review and Meta-Analysis According to PRISMA 2020 Guidelines: Results from Recently Published Papers in the Korean Journal of Radiology

OBJECTIVE

To evaluate the completeness of the reporting of systematic reviews and meta-analyses published in a general radiology journal using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines.

MATERIALS AND METHODS

Twenty-four articles (systematic review and meta-analysis, n = 18; systematic review only, n = 6) published between August 2009 and September 2021 in the Korean Journal of Radiology were analyzed. Completeness of the reporting of main texts and abstracts were evaluated using the PRISMA 2020 statement. For each item in the statement, the proportion of studies that met the guidelines' recommendation was calculated and items that were satisfied by fewer than 80% of the studies were identified. The review process was conducted by two independent reviewers.

RESULTS

Of the 42 items (including sub-items) in the PRISMA 2020 statement for main text, 24 were satisfied by fewer than 80% of the included articles. The 24 items were grouped into eight domains: 1) assessment of the eligibility of potential articles, 2) assessment of the risk of bias, 3) synthesis of results, 4) additional analysis of study heterogeneity, 5) assessment of non-reporting bias, 6) assessment of the certainty of evidence, 7) provision of limitations of the study, and 8) additional information, such as protocol registration. Of the 12 items in the abstract checklists, eight were incorporated in fewer than 80% of the included publications.

CONCLUSION

Several items included in the PRISMA 2020 checklist were overlooked in systematic review and meta-analysis articles published in the Korean Journal of Radiology. Based on these results, we suggest a double-check list for improving the quality of systematic reviews and meta-analyses. Authors and reviewers should familiarize themselves with the PRISMA 2020 statement and check whether the recommended items are fully satisfied prior to publication.