Diagnostic Performance of a Commercially Available Computer-aided Diagnosis System for Automatic Detection of Pulmonary Nodules: Comparison with Single and Double Reading

Predicting subsolid pulmonary nodules before percutaneous needle biopsy: a comparison of artificial neural network and biopsy results

AIM

To establish an artificial neural network (ANN) model to predict subsolid nodules (SSNs) before percutaneous core-needle biopsy (PCNB). The results of the two methods were compared to provide guidance on the treatment of SSNs.

MATERIALS AND METHODS

This was a single-centre retrospective study using data from 1,459 SSNs between 2013 and 2021. The ANN was developed using data from patients who underwent surgery following computed tomography (CT) (SFC) and validated using data from patients who underwent surgery following biopsy (SFB). The prediction results of the ANN for the PCNB group and the histopathological results obtained after biopsy were compared with the histopathological results of lung nodules in the same group after surgery. Additionally, the choice of predictors for PCNB was analysed using multivariate analysis.

RESULTS

There was no significant difference between the accuracies of the ANN and PCNB in the SFB group (p=0.086). The sensitivity of PCNB was lower than that of the ANN (p=0.000), but the specificity was higher (p=0.001). PCNB had better diagnostic ability than the ANN. The incidence of precursor lesions and non-neoplastic lesions in the SFB group was lower than that in the SFC group (p=0.000). A history of malignant tumours, size (2-3 cm), volume (>400 cm3) and mean CT value (≥-450 HU) are important factors for selecting PCNB.

CONCLUSIONS

Both ANN and PCNB have comparable accuracy in diagnosing SSNs; however, PCNB has a slightly higher diagnostic ability than ANN. Selecting appropriate patients for PCNB is important for maximising the benefit to SSN patients.

The use of artificial intelligence tools in cancer detection compared to the traditional diagnostic imaging methods: An overview of the systematic reviews

BACKGROUND AND PURPOSE

In comparison to conventional medical imaging diagnostic modalities, the aim of this overview article is to analyze the accuracy of the application of Artificial Intelligence (AI) techniques in the identification and diagnosis of malignant tumors in adult patients.

DATA SOURCES

The acronym PIRDs was used and a comprehensive literature search was conducted on PubMed, Cochrane, Scopus, Web of Science, LILACS, Embase, Scielo, EBSCOhost, and grey literature through Proquest, Google Scholar, and JSTOR for systematic reviews of AI as a diagnostic model and/or detection tool for any cancer type in adult patients, compared to the traditional diagnostic radiographic imaging model. There were no limits on publishing status, publication time, or language. For study selection and risk of bias evaluation, pairs of reviewers worked separately.

RESULTS

In total, 382 records were retrieved in the databases, 364 after removing duplicates, 32 satisfied the full-text reading criterion, and 09 papers were considered for qualitative synthesis. Although there was heterogeneity in terms of methodological aspects, patient differences, and techniques used, the studies found that several AI approaches are promising in terms of specificity, sensitivity, and diagnostic accuracy in the detection and diagnosis of malignant tumors. When compared to other machine learning algorithms, the Super Vector Machine method performed better in cancer detection and diagnosis. Computer-assisted detection (CAD) has shown promising in terms of aiding cancer detection, when compared to the traditional method of diagnosis.

CONCLUSIONS

The detection and diagnosis of malignant tumors with the help of AI seems to be feasible and accurate with the use of different technologies, such as CAD systems, deep and machine learning algorithms and radiomic analysis when compared with the traditional model, although these technologies are not capable of to replace the professional radiologist in the analysis of medical images. Although there are limitations regarding the generalization for all types of cancer, these AI tools might aid professionals, serving as an auxiliary and teaching tool, especially for less trained professionals. Therefore, further longitudinal studies with a longer follow-up duration are required for a better understanding of the clinical application of these artificial intelligence systems.

TRIAL REGISTRATION

Systematic review registration. Prospero registration number: CRD42022307403.

Assisted versus Manual Interpretation of Low-Dose CT Scans for Lung Cancer Screening: Impact on Lung-RADS Agreement

Purpose To compare the inter- and intraobserver agreement and reading times achieved when assigning Lung Imaging Reporting and Data System (Lung-RADS) categories to baseline and follow-up lung cancer screening studies by using a dedicated CT lung screening viewer with integrated nodule detection and volumetric support with those achieved by using a standard picture archiving and communication system (PACS)-like viewer. Materials and Methods Data were obtained from the National Lung Screening Trial (NLST). By using data recorded by NLST radiologists, scans were assigned to Lung-RADS categories. For each Lung-RADS category (1 or 2, 3, 4A, and 4B), 40 CT scans (20 baseline scans and 20 follow-up scans) were randomly selected for 160 participants (median age, 61 years; interquartile range, 58-66 years; 61 women) in total. Seven blinded observers independently read all CT scans twice in a randomized order with a 2-week washout period: once by using the standard PACS-like viewer and once by using the dedicated viewer. Observers were asked to assign a Lung-RADS category to each scan and indicate the risk-dominant nodule. Inter- and intraobserver agreement was analyzed by using Fleiss κ values and Cohen weighted κ values, respectively. Reading times were compared by using a Wilcoxon signed rank test. Results The interobserver agreement was moderate for the standard viewer and substantial for the dedicated viewer, with Fleiss κ values of 0.58 (95% CI: 0.55, 0.60) and 0.66 (95% CI: 0.64, 0.68), respectively. The intraobserver agreement was substantial, with a mean Cohen weighted κ value of 0.67. The median reading time was significantly reduced from 160 seconds with the standard viewer to 86 seconds with the dedicated viewer (P < .001). Conclusion Lung-RADS interobserver agreement increased from moderate to substantial when using the dedicated CT lung screening viewer. The median reading time was substantially reduced when scans were read by using the dedicated CT lung screening viewer. Keywords: CT, Thorax, Lung, Computer Applications-Detection/Diagnosis, Observer Performance, Technology Assessment Supplemental material is available for this article. © RSNA, 2021.

A comparison of the fusion model of deep learning neural networks with human observation for lung nodule detection and classification

OBJECTIVES

To compare the diagnostic performance of a newly developed artificial intelligence (AI) algorithm derived from the fusion of convolution neural networks (CNN) versus human observers in the estimation of malignancy risk in pulmonary nodules.

METHODS

The study population consists of 158 nodules from 158 patients. All nodules (81 benign and 77 malignant) were determined to be malignant or benign by a radiologist based on pathologic assessment and/or follow-up imaging. Two radiologists and an AI platform analyzed the nodules based on the Lung-RADS classification. The two observers also noted the size, location, and morphologic features of the nodules. An intraclass correlation coefficient was calculated for both observers and the AI; ROC curve analysis was performed to determine diagnostic performances.

RESULTS

Nodule size, presence of spiculation, and presence of fat were significantly different between the malignant and benign nodules (p < 0.001, for all three). Eighteen (11.3%) nodules were not detected and analyzed by the AI. Observer 1, observer 2, and the AI had an AUC of 0.917 ± 0.023, 0.870 ± 0.033, and 0.790 ± 0.037 in the ROC analysis of malignity probability, respectively. The observers were in almost perfect agreement for localization, nodule size, and lung-RADS classification [κ (95% CI)=0.984 (0.961-1.000), 0.978 (0.970-0.984), and 0.924 (0.878-0.970), respectively].

CONCLUSION

The performance of the fusion AI algorithm in estimating the risk of malignancy was slightly lower than the performance of the observers. Fusion AI algorithms might be applied in an assisting role, especially for inexperienced radiologists.

ADVANCES IN KNOWLEDGE

In this study, we proposed a fusion model using four state-of-art object detectors for lung nodule detection and discrimination. The use of fusion of deep learning neural networks might be used in a supportive role for radiologists when interpreting lung nodule discrimination.

Artificial intelligence for detection and characterization of pulmonary nodules in lung cancer CT screening: ready for practice?

Lung cancer computed tomography (CT) screening trials using low-dose CT have repeatedly demonstrated a reduction in the number of lung cancer deaths in the screening group compared to a control group. With various countries currently considering the implementation of lung cancer screening, recurring discussion points are, among others, the potentially high false positive rates, cost-effectiveness, and the availability of radiologists for scan interpretation. Artificial intelligence (AI) has the potential to increase the efficiency of lung cancer screening. We discuss the performance levels of AI algorithms for various tasks related to the interpretation of lung screening CT scans, how they compare to human experts, and how AI and humans may complement each other. We discuss how AI may be used in the lung cancer CT screening workflow according to the current evidence and describe the additional research that will be required before AI can take a more prominent role in the analysis of lung screening CT scans.

Added value of double reading in diagnostic radiology,a systematic review

Objectives

Double reading in diagnostic radiology can find discrepancies in the original report, but a systematic program of double reading is resource consuming. There are conflicting opinions on the value of double reading. The purpose of the current study was to perform a systematic review on the value of double reading.

Methods

A systematic review was performed to find studies calculating the rate of misses and overcalls with the aim of establishing the added value of double reading by human observers.

Results

The literature search resulted in 1610 hits. After abstract and full-text reading, 46 articles were selected for analysis. The rate of discrepancy varied from 0.4 to 22% depending on study setting. Double reading by a sub-specialist, in general, led to high rates of changed reports.

Conclusions

The systematic review found rather low discrepancy rates. The benefit of double reading must be balanced by the considerable number of working hours a systematic double-reading scheme requires. A more profitable scheme might be to use systematic double reading for selected, high-risk examination types. A second conclusion is that there seems to be a value of sub-specialisation for increased report quality. A consequent implementation of this would have far-reaching organisational effects.

Key Points

• In double reading, two or more radiologists read the same images.

• A systematic literature review was performed.

• The discrepancy rates varied from 0.4 to 22% in various studies.

• Double reading by sub-specialists found high discrepancy rates.

Electronic supplementary material

The online version of this article (10.1007/s13244-018-0599-0) contains supplementary material, which is available to authorised users.

Potential of computer-aided diagnosis to improve CT lung cancer screening

The development of low-dose spiral computed tomography (CT) has rekindled hope that effective lung cancer screening might yet be found. Screening is justified when there is evidence that it will extend lives at reasonable cost and acceptable levels of risk. A screening test should detect all extant cancers while avoiding unnecessary workups. Thus optimal screening modalities have both high sensitivity and specificity. Due to the present state of technology, radiologists must opt to increase sensitivity and rely on follow-up diagnostic procedures to rule out the incurred false positives. There is evidence in published reports that computer-aided diagnosis technology may help radiologists alter the benefit-cost calculus of CT sensitivity and specificity in lung cancer screening protocols. This review will provide insight into the current discussion of the effectiveness of lung cancer screening and assesses the potential of state-of-the-art computer-aided design developments.

Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume

Objective

To evaluate performance of computer-aided detection (CAD) beyond double reading for pulmonary nodules on low-dose computed tomography (CT) by nodule volume.

Methods

A total of 400 low-dose chest CT examinations were randomly selected from the NELSON lung cancer screening trial. CTs were evaluated by two independent readers and processed by CAD. A total of 1,667 findings marked by readers and/or CAD were evaluated by a consensus panel of expert chest radiologists. Performance was evaluated by calculating sensitivity of pulmonary nodule detection and number of false positives, by nodule characteristics and volume.

Results

According to the screening protocol, 90.9 % of the findings could be excluded from further evaluation, 49.2 % being small nodules (less than 50 mm³). Excluding small nodules reduced false-positive detections by CAD from 3.7 to 1.9 per examination. Of 151 findings that needed further evaluation, 33 (21.9 %) were detected by CAD only, one of them being diagnosed as lung cancer the following year. The sensitivity of nodule detection was 78.1 % for double reading and 96.7 % for CAD. A total of 69.7 % of nodules undetected by readers were attached nodules of which 78.3 % were vessel-attached.

Conclusions

CAD is valuable in lung cancer screening to improve sensitivity of pulmonary nodule detection beyond double reading, at a low false-positive rate when excluding small nodules.

Key Points

• Computer-aided detection (CAD) has known advantages for computed tomography (CT).

• Combined CAD/nodule size cut-off parameters assist CT lung cancer screening.

• This combination improves the sensitivity of pulmonary nodule detection by CT.

• It increases the positive predictive value for cancer detection.

Image analysis of small pulmonary nodules identified by computed tomography

Detection of small pulmonary nodules has markedly increased as computed tomography (CT) technology has advanced and interpretation evolved from viewing small CT images on film to magnified images on large, high-resolution computer monitors. Despite these advances, determining the etiology of a lung nodule short of major surgery remains problematic. Initial nodule size is a major criterion in evaluating the risk for malignancy, and the majority of CT detected nodules are <10 mm in diameter. Also, the likelihood that the nodule is a lung cancer increases with increasing age and smoking history, and such clinical information needs to be integrated into algorithms that guide the workup of such nodules. Baseline and annual repeat screening results are also very helpful in developing and assessing the usefulness of such algorithms. Based on CT morphology, subtypes of nodules have been identified; today nodules are routinely classified as being solid, part-solid, or nonsolid. It has been shown that part-solid nodules have a higher frequency of being malignant than solid or nonsolid ones. Other nodule characteristics such as spiculation are useful, although granulomas and fibrosis also have such features, so these characteristics have not been as useful as nodule-growth assessment. Depending on the aggressiveness of the lung cancer and the size of the nodule when it is initially seen, a follow-up CT scan 1-3 months after the first CT scan can identify those nodules with growth at a malignant rate. Software has been developed by all CT scanner manufacturers for such growth assessment, but the inherent variability of such assessments needs further development. Nodule-growth assessment based on 2-dimensional approaches is limited; therefore, software has been developed for the 3-dimensional assessment of growth. Different approaches for such growth assessment have been developed, either using automated computer segmentation techniques or hybrid methods that allow the radiologist to adjust such segmentation. There are, however, inherent reasons for variability in such measurements that need to be carefully considered, and this, together with continued technologic advances and integration of the relevant clinical information, will allow for individualization of the algorithms for the workup of small pulmonary nodules.

The use of pre-operative computed tomography in the assessment of the acute abdomen

INTRODUCTION

While there are a lot of data on the accuracy of computed tomography (CT) in diagnosing specific causes of an acute abdomen, there is very little information on the accuracy of CT in the acute general surgical admissions workload. We look at the diagnostic accuracy of CT in patients presenting with an acute abdomen who ultimately required a laparotomy.

METHODS

Patients who underwent an emergency laparotomy between 2008 and 2010 at Eastbourne District General Hospital with pre-operative CT on the same admission were included in the study. The CT report was compared with the laparotomy and histology findings and, where a discrepancy existed, the original imaging was reviewed by a senior consultant blinded to the original report and laparotomy findings.

RESULTS

A total of 196 emergency laparotomies were performed over the 2-year period, with 112 patients undergoing preoperative CT. Fifteen patients were excluded from the study due to missing notes. In the remaining 97 patients, 80 CT reports correlated with the final operative diagnosis, giving a diagnostic accuracy of 82%. Of these, the on-call registrar was the initial reporter in 37 scans, with a diagnostic accuracy of 78%. On review of the CT by a second consultant, this increased to 90 correlations, yielding an accuracy of 93%. Delay between CT and the operation did not significantly alter diagnostic accuracy, nor was there any statistically significant reduction in accuracy in reports issued by on-call registrars.

CONCLUSIONS

On first reporting, CT misses 18% of diagnoses that ultimately require operative intervention. Reducing the threshold for obtaining a second consultant radiologist review significantly improves the diagnostic accuracy to 93%.

Recent technological and application developments in computed tomography and magnetic resonance imaging for improved pulmonary nodule detection and lung cancer staging

This review compares the emerging technologies and approaches in the application of magnetic resonance (MR) and computed tomography (CT) imaging for the assessment of pulmonary nodules and staging of malignant findings. Included in this review is a brief definition of pulmonary nodules and an introduction to the challenges faced. We have highlighted the current status of both MR and CT for the early detection of lung nodules. Developments are detailed in this review for the management of pulmonary nodules using advanced imaging, including: dynamic imaging studies, dual energy CT, computer aided detection and diagnosis, and imaging assisted nodule biopsy approaches which have improved lung nodule detection and diagnosis rates. Recent advancements linking in vivo imaging to corresponding histological pathology are also highlighted. In vivo imaging plays a pivotal role in the clinical staging of pulmonary nodules through TNM assessment. While CT and positron emission tomography (PET)/CT are currently the most commonly clinically employed modalities for pulmonary nodule staging, studies are presented that highlight the augmentative potential of MR.

Computer-aided detection of lung nodules: influence of the image reconstruction kernel for computer-aided detection performance

OBJECTIVE

To evaluate the relationship between a computed tomographic reconstruction kernel and the sensitivity of a computer-aided detection (CAD) system for lung nodule detection.

METHODS

We retrospectively studied 36 consecutive patients with no known pulmonary nodules who underwent low-dose computed tomography for lung cancer screening with 3 different reconstruction kernels (B, C, and L). All series were reviewed with a commercial CAD system for lung nodule detection.

RESULTS

The 36 scans showed 231 uncalcified nodules (170 micronodules and 61 nodules). There was little variation of sensitivities for each series (82%, 88%, and 82% for the nodules of B, C, and L, respectively). When the results of 2 series were combined, sensitivities were boosted (B + C, 89%; B + L, 95%; and C + L, 96% for the nodules).

CONCLUSIONS

Sensitivity of the CAD system was influenced by the selection of the reconstruction kernel. By combining data from 2 different kernels, CAD sensitivity can be elevated without further patient radiation exposure.

A process model for direct correlation between computed tomography and histopathology application in lung cancer

RATIONALE AND OBJECTIVES

Multimodal imaging techniques for capturing normal and diseased human anatomy and physiology are being developed to benefit patient clinical care, research, and education. In the past, the incorporation of histopathology into these multimodal datasets has been complicated by the large differences in image quality, content, and spatial association.

MATERIALS AND METHODS

We have developed a novel system, the large-scale image microtome array (LIMA), to bridge the gap between nonstructurally destructive and destructive imaging such that reliable registration between radiological data and histopathology can be achieved. Registration algorithms have been designed to align the multimodal datasets, which include computed tomography, computed micro-tomography, LIMA, and histopathology data to a common coordinate system.

RESULTS

The resulting volumetric dataset provides an abundance of valuable information relating to the tissue sample including density, anatomical structure, color, texture, and cellular information in three dimensions. An image processing pipeline has been established to register all the multimodal data to a common coordinate system.

CONCLUSION

In this study, we have chosen to use human lung cancer nodules as an example; however, the flexibility of the image acquisition and subsequent processing algorithms makes it applicable to any soft organ tissue. A novel process model has been established to generate cross registered multimodal datasets for the investigation of human lung cancer nodule content and associated image-based representation.

Effect of slab thickness on the CT detection of pulmonary nodules: use of sliding thin-slab maximum intensity projection and volume rendering

OBJECTIVE

The objective of this study was to evaluate the detection rates of pulmonary nodules on CT as a function of slab thickness using sliding thin-slab maximum intensity projection (MIP) and volume rendering (VR).

SUBJECTS AND METHODS

Eighty-eight oncology patients (33 women, 55 men; mean age, 59 years; age range, 18-81 years) who routinely underwent chest CT examinations were prospectively included. Two radiologists independently evaluated each CT examination for the presence of pulmonary nodules using MIP and VR, with each image reconstructed using three different slab thicknesses (5, 8, 11 mm). The standard of reference was the maximum number of detected nodules, which were classified by localization and size, judged to be true-positives by a consensus panel. Interreader agreement was assessed by kappa value on a nodule-by-nodule basis. Sensitivities for both reconstruction techniques and for the three slab thicknesses were calculated using the proportion procedure for survey data with the patient as the primary sample unit and were compared using the Wilcoxon's signed rank test with Bonferroni correction for both readers separately.

RESULTS

One thousand fifty-eight true-positive nodules were detected. Interreader agreement was fair to moderate. Sensitivity for pulmonary nodules was superior for 8-mm MIP (reader 1, 84%; reader 2, 81%) and was significantly better than the sensitivities of all other tested techniques for both readers (p < 0.001 each) independent of nodule localization and size (except for one reader's analysis of 8-mm MIP versus 11-mm MIP for nodules > 8 mm). A higher sensitivity was achieved using MIP than VR.

CONCLUSION

MIP with a slab thickness of 8 mm is superior in the detection of pulmonary nodules to all other tested techniques.

Evaluation of computer-aided diagnosis (CAD) software for the detection of lung nodules on multidetector row computed tomography (MDCT): JAFROC study for the improvement in radiologists' diagnostic accuracy

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the usefulness of computer-aided diagnosis (CAD) software for the detection of lung nodules on multidetector-row computed tomography (MDCT) in terms of improvement in radiologists' diagnostic accuracy in detecting lung nodules, using jackknife free-response receiver-operating characteristic (JAFROC) analysis.

MATERIALS AND METHODS

Twenty-one patients (6 without and 15 with lung nodules) were selected randomly from 120 consecutive thoracic computed tomographic examinations. The gold standard for the presence or absence of nodules in the observer study was determined by consensus of two radiologists. Six expert radiologists participated in a free-response receiver operating characteristic study for the detection of lung nodules on MDCT, in which cases were interpreted first without and then with the output of CAD software. Radiologists were asked to indicate the locations of lung nodule candidates on the monitor with their confidence ratings for the presence of lung nodules.

RESULTS

The performance of the CAD software indicated that the sensitivity in detecting lung nodules was 71.4%, with 0.95 false-positive results per case. When radiologists used the CAD software, the average sensitivity improved from 39.5% to 81.0%, with an increase in the average number of false-positive results from 0.14 to 0.89 per case. The average figure-of-merit values for the six radiologists were 0.390 without and 0.845 with the output of the CAD software, and there was a statistically significant difference (P < .0001) using the JAFROC analysis.

CONCLUSION

The CAD software for the detection of lung nodules on MDCT has the potential to assist radiologists by increasing their accuracy.

[Usefulness of maximum intensity projections in low-radiation multislice CT lung cancer screening]

OBJECTIVE

To evaluate the diagnostic accuracy of non-overlapping 10-mm-thick axial maximum intensity projections (MIP) in the detection of pulmonary nodules in subjects participating in a lung cancer screening program (LCSP) using multislice computed tomography (MSCT) with a low dose of radiation.

MATERIAL AND METHODS

We evaluated 52 consecutive low-radiation MSCT studies in asymptomatic smokers included in an LCSP (1.25 mm axial images). Axial MIPs with 10mm slice thickness (30 images) were performed and evaluated retrospectively; readers were blind to the initial radiological report. All nodules detected were considered, regardless of their size or consistency. The standard of reference was determined by double reading and consensus for each nodule.

RESULTS

A total of 162 pulmonary nodules (mean size: 3.9 mm, sd: 1.7) were detected. MIP reconstruction detected 150 nodules (S = 92.6%). The initial radiological evaluation detected 108 nodules (S = 66.7%). MIP reconstruction detected 54 (33.3%) nodules that were not reported initially (mean size: 3.4 mm; sd: 1.2) but failed to detect 12 (7.4%) of the nodules reported initially (mean size: 2.91 mm; sd: 0.8). MIP detected all 35 nodules > or = 5 mm, (S =100), whereas the initial radiological evaluation only detected 27 (S = 77%). MIP reconstruction enabled more of the nodules to be detected than the 1.25-mm conventional axial slices (p < 0.01).

CONCLUSION

The introduction of non-overlapping 10-mm-thick axial MIP reconstructions in a low-radiation LCSP using MSCT enabled nodules more accurate and faster detection of pulmonary nodules in comparison with 1.25 mm conventional axial slices.

Preoperative imaging for metastasectomy

For most solid neoplasms, medical imaging is a vital component of tumor staging and surveillance. Imaging strategies vary according to the type and grade of primary neoplasm, tumor stage at diagnosis, tumor markers, previous therapies, and patient symptoms. In this article, we address imaging of individual organs (lung, liver, adrenals) and outline imaging strategies for specific types of neoplasms.

The Lung Image Database Consortium (LIDC): ensuring the integrity of expert-defined "truth"

RATIONALE AND OBJECTIVES

Computer-aided diagnostic (CAD) systems fundamentally require the opinions of expert human observers to establish "truth" for algorithm development, training, and testing. The integrity of this "truth," however, must be established before investigators commit to this "gold standard" as the basis for their research. The purpose of this study was to develop a quality assurance (QA) model as an integral component of the "truth" collection process concerning the location and spatial extent of lung nodules observed on computed tomography (CT) scans to be included in the Lung Image Database Consortium (LIDC) public database.

MATERIALS AND METHODS

One hundred CT scans were interpreted by four radiologists through a two-phase process. For the first of these reads (the "blinded read phase"), radiologists independently identified and annotated lesions, assigning each to one of three categories: "nodule >or=3 mm," "nodule <3 mm," or "non-nodule >or=3 mm." For the second read (the "unblinded read phase"), the same radiologists independently evaluated the same CT scans, but with all of the annotations from the previously performed blinded reads presented; each radiologist could add to, edit, or delete their own marks; change the lesion category of their own marks; or leave their marks unchanged. The post-unblinded read set of marks was grouped into discrete nodules and subjected to the QA process, which consisted of identification of potential errors introduced during the complete image annotation process and correction of those errors. Seven categories of potential error were defined; any nodule with a mark that satisfied the criterion for one of these categories was referred to the radiologist who assigned that mark for either correction or confirmation that the mark was intentional.

RESULTS

A total of 105 QA issues were identified across 45 (45.0%) of the 100 CT scans. Radiologist review resulted in modifications to 101 (96.2%) of these potential errors. Twenty-one lesions erroneously marked as lung nodules after the unblinded reads had this designation removed through the QA process.

CONCLUSIONS

The establishment of "truth" must incorporate a QA process to guarantee the integrity of the datasets that will provide the basis for the development, training, and testing of CAD systems.

The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans

RATIONALE AND OBJECTIVES

The purpose of this study was to analyze the variability of experienced thoracic radiologists in the identification of lung nodules on computed tomography (CT) scans and thereby to investigate variability in the establishment of the "truth" against which nodule-based studies are measured.

MATERIALS AND METHODS

Thirty CT scans were reviewed twice by four thoracic radiologists through a two-phase image annotation process. During the initial "blinded read" phase, radiologists independently marked lesions they identified as "nodule >or=3 mm (diameter)," "nodule <3 mm," or "non-nodule >or=3 mm." During the subsequent "unblinded read" phase, the blinded read results of all four radiologists were revealed to each radiologist, who then independently reviewed their marks along with the anonymous marks of their colleagues; a radiologist's own marks then could be deleted, added, or left unchanged. This approach was developed to identify, as completely as possible, all nodules in a scan without requiring forced consensus.

RESULTS

After the initial blinded read phase, 71 lesions received "nodule >or=3 mm" marks from at least one radiologist; however, all four radiologists assigned such marks to only 24 (33.8%) of these lesions. After the unblinded reads, a total of 59 lesions were marked as "nodule >or=3 mm" by at least one radiologist. Twenty-seven (45.8%) of these lesions received such marks from all four radiologists, three (5.1%) were identified as such by three radiologists, 12 (20.3%) were identified by two radiologists, and 17 (28.8%) were identified by only a single radiologist.

CONCLUSION

The two-phase image annotation process yields improved agreement among radiologists in the interpretation of nodules >or=3 mm. Nevertheless, substantial variability remains across radiologists in the task of lung nodule identification.

Interobserver-variability of lung nodule volumetry considering different segmentation algorithms and observer training levels

OBJECTIVE

The aim of this study was to investigate the interobserver variability of CT based diameter and volumetric measurements of artificial pulmonary nodules. A special interest was the consideration of different measurement methods, observer experience and training levels.

MATERIALS AND METHODS

For this purpose 46 artificial small solid nodules were examined in a dedicated ex-vivo chest phantom with multislice-spiral CT (20 mAs, 120 kV, collimation 16 mm x 0.75 mm, table feed 15 mm, reconstructed slice thickness 1mm, reconstruction increment 0.7 mm, intermediate reconstruction kernel). Two observer groups of different radiologic experience (0 and more than 5 years of training, 3 observers each) analysed all lesions with digital callipers and 2 volumetry software packages (click-point depending and robust volumetry) in a semi-automatic and manually corrected mode. For data analysis the variation coefficient (VC) was calculated in per cent for each group and a Wilcoxon test was used for analytic statistics.

RESULTS

Click-point robust volumetry showed with a VC of <0.01% in both groups the smallest interobserver variability. Between experienced and un-experienced observers interobserver variability was significantly different for diameter measurements (p=0.023) but not for semi-automatic and manual corrected volumetry. A significant training effect was revealed for diameter measurements (p=0.003) and semi-automatic measurements of click-point depending volumetry (p=0.007) in the un-experienced observer group.

CONCLUSIONS

Compared to diameter measurements volumetry achieves a significantly smaller interobserver variance and advanced volumetry algorithms are independent of observer experience.

Pulmonary nodule detection on MDCT images: evaluation of diagnostic performance using thin axial images, maximum intensity projections, and computer-assisted detection

This study aimed at evaluating the diagnostic benefits of maximum intensity projections (MIP) and a commercially available computed-assisted detection system (CAD) for the detection of pulmonary nodules on MDCT as compared with standard 1-mm images on lung cancer screening material. Thirty subjects were randomly selected from our database. Three radiologists independently reviewed three types of images: axial 1-mm images, axial MIP slabs, and CAD system detections. Two independent experienced chest radiologists decided which were true-positive nodules. Two hundred eighty-five nodules > or =1 mm were identified as true-positive by consensus of two independent chest radiologists. The detection rates of the three independent observers with 1-mm axial images were 22 +/- 4.8%, 30 +/- 5.3%, and 47 +/- 2.8%; with MIP: 33 +/- 5.4%, 39 +/- 5.7%, and 45 +/- 5.8%; and with CAD: 35 +/- 5.6%, 36 +/- 5.6%, and 36 +/- 5.6%. There was a reading technique effect on the observers' sensitivity for nodule detection: sensitivities with MIP were higher than with 1-mm images or CAD for all nodules (F-values = 0.046). For nodules > or =3 mm, readers' sensitivities were higher with 1-mm images or MIP than with CAD (p < 0.0001). CAD was the most and MIP the less time-consuming technique (p < 0.0001). MIP and CAD reduced the number of overlooked small nodules. As MIP is more sensitive and less time consuming than the CAD we used, we recommend viewing MIP and 1-mm images for the detection of pulmonary nodules.

[Computer-aided detection of lung nodules on thin collimation MDCT: impact on radiologists' performance]

OBJECTIVES

Evaluate the improvement in detecting lung nodules when using multidetector CT (MDCT) computer-assisted diagnosis (CAD).

MATERIAL AND METHODS

Three radiologists (R1, R2, R3) with different levels of experience independently interpreted 30 MDCT examinations of the thorax taken for screening purposes, first without and then with CAD. The diagnosis was established by two of the three radiologists interpreting the images together, assisted by the CAD.

RESULTS

The consensus reading identified 133 nodules, 61 (46%) of which were 4 mm or larger. The sensitivity values in the detection of nodules before and after using the CAD were 54% and 80% (R1), 38% and 71% (R2), and 70% and 88% (R3), respectively. When considering only the nodules that were 4 mm or larger, the sensitivity values varied before and after using the CAD, from 62% to 95% (R1), from 41% to 84% (R2), and from 74% to 92% (R3). By combining two by two the three radiologists' results obtained without the CAD, the sensitivity values were 65%, 83%, and 77%, respectively, for all the nodules, and 70%, 85%, and 77% for the nodules that were 4 mm or larger. The CAD induced a total of 105 false-positive results, with a mean of 3.5 per examination.

CONCLUSION

The lung nodules missed by the radiologist can be detected if the CAD is used as a second reader. The CAD can be at least as beneficial as the use of a second independent reader.

Precision of computer-aided volumetry of artificial small solid pulmonary nodules inex vivoporcine lungs

The purpose of this study was to investigate the precision of CT-based volumetric measurements of artificial small pulmonary nodules under ex vivo conditions. We implanted 322 artificial nodules in 23 inflated ex vivo porcine lungs in a dedicated chest phantom. The lungs were examined with a multislice spiral CT (20 mAs, collimation 16x0.75 mm, 1 mm slice thickness, 0.7 mm increment). A commercial volumetry software package (LungCARE VA70C-W; Siemens, Erlangen, Germany) was used for volume analysis in a semi-automatic and a manual corrected mode. After imaging, the lungs were dissected to harvest the nodules for gold standard determination. The volumes of 202 solitary, solid and well-defined lesions without contact with the pleura, greater bronchi or vessels were compared with the results of volumetry. A mean nodule diameter of 8.3 mm (+/-2.1 mm) was achieved. The mean relative deviation from the true lesion volume was -9.2% (+/-10.6%) for semi-automatic and -0.3% (+/-6.5%) for manual corrected volumetry. The subgroup of lesions from 5 mm to <10 mm in diameter showed a mean relative deviation of -8.7% (+/-10.9%) for semi-automatic volumetry and -0.3% (+/-6.9%) for manually corrected volumetry. We conclude that the presented software allowed for precise volumetry of artificial nodules in ex vivo lung tissue. This result is comparable to the findings of previous in vitro studies.

Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader

The purpose of this study was to compare sensitivity for detection of pulmonary nodules in MDCT scans and reading time of radiologists when using CAD as the second reader (SR) respectively concurrent reader (CR). Four radiologists analyzed 50 chest MDCT scans chosen from clinical routine two times and marked all detected pulmonary nodules: first with CAD as CR (display of CAD results immediately in the reading session) and later (median 14 weeks) with CAD as SR (display of CAD markers after completion of first reading without CAD). A Siemens LungCAD prototype was used. Sensitivities for detection of nodules and reading times were recorded. Sensitivity of reading with CAD as SR was significantly higher than reading without CAD (p < 0.001) and CAD as CR (p < 0.001). For nodule size of 1.75 mm or above no significant sensitivity difference between CAD as CR and reading without CAD was observed; e.g., for nodules above 4 mm sensitivity was 68% without CAD, 68% with CAD as CR (p = 0.45) and 75% with CAD as SR (p < 0.001). Reading time was significantly shorter for CR (274 s) compared to reading without CAD (294 s; p = 0.04) and SR (337 s; p < 0.001). In our study CAD could either speed up reading of chest CT cases for pulmonary nodules without relevant loss of sensitivity when used as CR, or it increased sensitivity at the cost of longer reading times when used as SR.

Volumetric measurements of lung nodules with multi-detector row CT: effect of changes in lung volume

Objective

To evaluate how changes in lung volume affect volumetric measurements of lung nodules using a multi-detector row CT.

Materials and Methods

Ten subjects with asthma or chronic bronchitis who had one or more lung nodules were included. For each subject, two sets of CT images were obtained at inspiration and at expiration. A total of 33 nodules (23 nodules ≥ 3 mm) were identified and their volume measured using a semiautomatic volume measurement program. Differences between nodule volume on inspiration and expiration were compared using the paired t-test. Percent differences, between on inspiration and expiration, in nodule attenuation, total lung volume, whole lung attenuation, and regional lung attenuation, were computed and compared with percent difference in nodule volume determined by linear correlation analysis.

Results

The difference in nodule volume observed between inspiration and expiration was significant (p < 0.01); the mean percent difference in lung nodule volume was 23.1% for all nodules and for nodules ≥ 3 mm. The volume of nodules was measured to be larger on expiration CT than on inspiration CT (28 out of 33 nodules; 19 out of 23 nodules ≥ 3 mm). A statistically significant correlation was found between the percent difference of lung nodule volume and lung volume or regional lung attenuation (p < 0.05) for nodules ≥ 3 mm.

Conclusion

Volumetric measurements of pulmonary nodules were significantly affected by changes in lung volume. The variability in this respiration-related measurement should be considered to determine whether growth has occurred in a lung nodule.

Small Pulmonary Nodules: Effect of Two Computer-aided Detection Systems on Radiologist Performance

PURPOSE

To prospectively compare the effects of two computer-aided detection (CAD) systems on the detection of small pulmonary nodules at multi-detector row computed tomography (CT) by using a consensus panel decision as the reference standard.

MATERIALS AND METHODS

Institutional review board approval and informed consent were obtained. Multi-detector row CT scans were randomly chosen and prospectively evaluated in 25 patients. Two dedicated CAD systems-ImageChecker CT (R2 Technologies, Sunnyvale, Calif) and Nodule Enhanced Viewing (NEV) (Siemens Medical Solutions, Forchheim, Germany)-were used. Results were interpreted by three radiologists with 1, 3, and 6 years of experience. Images were evaluated without and with CAD software. The reference standard was assessed by a consensus panel consisting of all three radiologists and an adjudicator with 8 years of experience.

RESULTS

A total of 116 pulmonary nodules (average diameter, 3.4 mm; average volume, 32.05 mm3) were found in all data sets during consensus interpretation, which included findings from the CAD software and all radiologists. Overall sensitivity was 73% with ImageChecker CT and 75% with NEV. Overall sensitivity without CAD was 68% for radiologist 1, 78% for radiologist 2, and 82% for radiologist 3. With ImageChecker CT, sensitivity increased to 79% for radiologist 1, 90% for radiologist 2, and 84% for radiologist 3. With NEV, sensitivity increased to 79% for radiologist 1, 90% for radiologist 2, and 86% for radiologist 3. The average number of false-positive findings was six (range, 0-14) with ImageChecker CT and eight (range, 0-22) with NEV.

CONCLUSION

Radiologist performance in the interpretation of multi-detector row CT scans can be improved by using CAD systems, with a reduction in the number of false-negative diagnoses. No statistically significant difference in sensitivity was found between the two CAD systems.

Solitary pulmonary nodule: detection and management

Pulmonary nodules are commonly detected at computed tomography (CT) of the chest. More than 95% are \documentclass[12pt]{minimal} \usepackage{wasysym} \usepackage[substack]{amsmath} \usepackage{amsfonts,amssymb,amsbsy} \usepackage[mathscr]{eucal} \usepackage{mathrsfs} \DeclareFontFamily{T1}{linotext}{} \DeclareFontShape{T1}{linotext}{m}{n}{<-> linotext}{} \DeclareSymbolFont{linotext}{T1}{linotext}{m}{n} \DeclareSymbolFontAlphabet{\mathLINOTEXT}{linotext} \begin{document} $\le $ \end{document} 10 mm; of these more than 95% are benign. Visual detection of pulmonary nodules by human readers is suboptimal, particularly with small nodules \documentclass[12pt]{minimal} \usepackage{wasysym} \usepackage[substack]{amsmath} \usepackage{amsfonts,amssymb,amsbsy} \usepackage[mathscr]{eucal} \usepackage{mathrsfs} \DeclareFontFamily{T1}{linotext}{} \DeclareFontShape{T1}{linotext}{m}{n}{<-> linotext}{} \DeclareSymbolFont{linotext}{T1}{linotext}{m}{n} \DeclareSymbolFontAlphabet{\mathLINOTEXT}{linotext} \begin{document} $\le $ \end{document} 10 mm. Computer-assisted detection can improve sensitivity and diagnostic confidence. Due to the high proportion of malignant lesions in nodules >10 mm immediate, often invasive work-up is required including contrast-enhanced dynamic CT, positron emission tomography (PET) or biopsy. However, in nodules \documentclass[12pt]{minimal} \usepackage{wasysym} \usepackage[substack]{amsmath} \usepackage{amsfonts,amssymb,amsbsy} \usepackage[mathscr]{eucal} \usepackage{mathrsfs} \DeclareFontFamily{T1}{linotext}{} \DeclareFontShape{T1}{linotext}{m}{n}{<-> linotext}{} \DeclareSymbolFont{linotext}{T1}{linotext}{m}{n} \DeclareSymbolFontAlphabet{\mathLINOTEXT}{linotext} \begin{document} $\le $ \end{document} 10 mm the high proportion of benign lesions requires a non-invasive work-up usually based on follow-up with unenhanced CT. Invasive procedures are only required for growing nodules. Stable nodules require further follow-up and decreasing nodules are considered benign.

Computer-aided detection and automated CT volumetry of pulmonary nodules

With use of multislice computed tomography (MSCT), small pulmonary nodules are being detected in vast numbers, constituting the majority of all noncalcified lung nodules. Although the prevalence of lung cancers among such lesions in lung cancer screening populations is low, their isolation may contribute to increased patient survival. Computer-aided diagnosis (CAD) has emerged as a diverse set of diagnostic tools to handle the large number of images in MSCT datasets and most importantly, includes automated detection and volumetry of pulmonary nodules. Current CAD systems can significantly enhance experienced radiologists' performance and outweigh human limitations in identifying small lesions and manually measuring their diameters, augment observer consistency in the interpretation of such examinations and may thus help to detect significantly higher rates of early malignomas and give more precise estimates on chemotherapy response than can radiologists alone. In this review, we give an overview of current CAD in lung nodule detection and volumetry and discuss their relative merits and limitations.

Measures of response: RECIST, WHO, and new alternatives

RECIST (Response Evaluation Criteria in Solid Tumors) is a widely employed method introduced in 2000 to assess change in tumor size in response to therapy. The simplicity of the technique, however, contrasts sharply with the increasing sophistication of imaging instrumentation. Anatomically based imaging measurement, although supportive of drug development and key to some accelerated drug approvals, is being pressed to improve its methodologic robustness, particularly in the light of more functionally-based imaging that is sensitive to tissue molecular response such as fluorodeoxyglucose positron emission tomography. Nevertheless ready availability of computed tomography and magnetic resonance imaging machines largely assures anatomically based imaging a continuing role in clinical trials for the foreseeable future. Recent advances in image processing enabled by the computational power of modern clinical scanners open a considerable opportunity to characterize tumor response to therapy as a complement to image acquisition. Various alternative quantitative volumetric approaches have been proposed but have yet to gain wide acceptance by clinical and regulatory communities, nor have these more complex techniques shown incontrovertible evidence of greater reproducibility or predictive value of clinical events and outcome. Unless plans are created for clinical trials that incorporate the design needed to prove the added value and unique clinical utility of these novel approaches, any theoretical benefit of these more elaborate methods could remain unfulfilled.

Quantification of lung tumor volume and rotation at 3D dynamic parallel MR imaging with view sharing: preliminary results

The purpose of this study was to prospectively evaluate the volumes and rotations of pulmonary nodules during respiration by using three-dimensional fast low-angle shot dynamic magnetic resonance (MR) imaging (1.5/0.6 [repetition time msec/echo time msec], 3.8 x 3.8 x 3.8-mm voxel size, imaging time per three-dimensional data set of 1 second). The feasibility of the technique was verified by using 130-, 40-, and 12-cm3 phantoms made of meatballs and in five patients with solitary intrapulmonary tumors (four men, one woman; median age, 60 years) at computed tomography and histologic analysis. All patients provided written informed consent, and the study was institutional review board approved. It was proved that there were no substantial differences among the 21 algorithms used to correct partial volume effects. The most precise algorithm (r > 0.9, P < .01) used to correct partial volume effects--with which mean phantom volumes of 120.8 cm3 +/- 4.1, 36.1 cm3 +/- 3.98, and 13.1 cm3 +/- 1.5 were calculated--yielded a root mean square error of 14%. The MR imaging-derived nodule volume and rotation during respiration could be quantified by using oriented bounding box techniques.

Reproducibility of computer-aided volumetry of artificial small pulmonary nodules in ex vivo porcine lungs

OBJECTIVE

The main purpose of this study was to investigate the reproducibility of computed tomography (CT)-based volumetric measurements of small pulmonary nodules.

METHODS

We implanted 70 artificial pulmonary nodules in 5 ex vivo porcine lungs in a dedicated chest phantom. The lungs were scanned 5 times consecutively with multislice-CT (collimation 16 x 0.75 mm, slice thickness 1 mm, reconstruction increment 0.7 mm). A commercial software package was used for lesion volumetry. The authors differentiated between intrascan reproducibility, interscan reproducibility, and results from semiautomatic and postprocessed volumetry.

RESULTS

Analysis of intrascan reproducibility revealed a mean variation coefficient of 6.2% for semiautomatic volumetry and of 0.7% for human adapted volumetry. For interscan reproducibility a mean variation coefficient of 9.2% and for human adapted volumetry a mean of 3.7% was detected.

CONCLUSION

The presented volumetry software showed a high reproducibility that can be expected to detect nodule growth with a high degree of certainty.

A proposed diagnostic approach to the patient with the subcentimeter pulmonary nodule: techniques that facilitate video-assisted thoracic surgery excision

This article describes a diagnostic approach to the management of subcentimeter pulmonary nodules and delineates the contribution the chest surgeon brings to clinical decision making. This includes clinical experience and knowledge of nodule accessibility to thoracoscopic excisional biopsy based on the anticipated localization technique to be used. Characteristics of the ideal localization technique are discussed. Different localization techniques are then described, and their respective advantages and disadvantages are discussed. These techniques include intraoperative finger and instrument localization, preoperative radiologically placed hooks and coils, intraoperative ultrasonography, and preoperative placement of radiotracer markers.

Lung Nodule Detection and Characterization With Multi-Slice CT

The influence of MSCT on nodule detection and characterization will be discussed. The objective is to improve understanding of the clinical issues involved in nodule detection, characterization, and management in light of technological advances. Topics to be covered are noninvasive characterization techniques, such as morphologic and density inspection on CT, nodule enhancement techniques, CT-PET, temporal nodule size assessment, and computer aided diagnosis for both detection and characterization.

New Developments in Lung Cancer Screening

Lung cancer is the most lethal cancer in our society. Late diagnosis of this disease is a major problem and so recent favorable reports with spiral computed tomography screening of high-risk populations have rekindled interest in improving early lung cancer detections. The process of lung cancer screening is a complicated process that involves many component activities. Interest to date has heavily focused on the initial case identification, but more recent reports have suggested that the issues with case work-up and surgical management also bear closer consideration. Given the dynamic nature of spiral computed tomography scan development and the remarkable improvements in imaging resolution over the last decade, there is an urgent need for research to establish optimal clinical management of early lung cancer detected in a screening setting.

Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT

The purpose of this study was to assess the effectiveness of double reading to increase the sensitivity of lung nodule detection at standard-dose (SDCT) and low-dose multirow-detector CT (LDCT). SDCT (100 mAs effective tube current) and LDCT (20 mAs) of nine patients with pulmonary metastases were obtained within 5 min using four-row detector CT. Softcopy images reconstructed with 5-mm slice thickness were read by three radiologists independently. Images with 1.25-mm slice thickness served as the gold standard. Sensitivity was assessed for single readers and combinations. The effectiveness of double reading was expressed as the increase of sensitivity. Average sensitivity for detection of 390 nodules (size 3.9+/-3.2 mm) for single readers was 0.63 (SDCT) and 0.64 (LDCT). Double reading significantly increased sensitivity to 0.74 and 0.79, respectively. No significant difference between sensitivity at SDCT and LDCT was observed. The percentage of nodules detected by all three readers concordantly was 52% for SDCT and 47% for LDCT. Although double reading increased the detection rate of pulmonary nodules from 63% to 74-79%, a considerable proportion of nodules remained undetected. No difference between sensitivities at LDCT and SDCT for detection of small nodules was observed.