Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists

Validation of a commercially available CAD-system for lung nodule detection and characterization using CT-scans

OBJECTIVES

This study aims to externally validate a commercially available Computer-Aided Detection (CAD)-system for the automatic detection and characterization of solid, part-solid, and ground-glass lung nodules (LN) on CT scans.

METHODS

This retrospective study encompasses 263 chest CT scans performed between January 2020 and December 2021 at a Dutch university hospital. All scans were read by a radiologist (R1) and compared with the initial radiology report. Conflicting scans were assessed by an adjudicating radiologist (R2). All scans were also processed by CAD. The standalone performance of CAD in terms of sensitivity and false-positive (FP)-rate for detection was calculated together with the sensitivity for characterization, including texture, calcification, speculation, and location. The R1's detection sensitivity was also assessed.

RESULTS

A total of 183 true nodules were identified in 121 nodule-containing scans (142 non-nodule-containing scans), of which R1 identified 165/183 (90.2%). CAD detected 149 nodules, of which 12 were not identified by R1, achieving a sensitivity of 149/183 (81.4%) with an FP-rate of 49/121 (0.405). CAD's detection sensitivity for solid, part-solid, and ground-glass LNs was 82/94 (87.2%), 42/47 (89.4%), and 25/42 (59.5%), respectively. The classification accuracy for solid, part-solid, and ground-glass LNs was 81/82 (98.8%), 16/42 (38.1%), and 18/25 (72.0%), respectively. Additionally, CAD demonstrated overall classification accuracies of 137/149 (91.9%), 123/149 (82.6%), and 141/149 (94.6%) for calcification, spiculation, and location, respectively.

CONCLUSIONS

Although the overall detection rate of this system slightly lags behind that of a radiologist, CAD is capable of detecting different LNs and thereby has the potential to enhance a reader's detection rate. While promising characterization performances are obtained, the tool's performance in terms of texture classification remains a subject of concern.

CLINICAL RELEVANCE STATEMENT

Numerous lung nodule computer-aided detection-systems are commercially available, with some of them solely being externally validated based on their detection performance on solid nodules. We encourage researchers to assess performances by incorporating all relevant characteristics, including part-solid and ground-glass nodules.

KEY POINTS

Few computer-aided detection (CAD) systems are externally validated for automatic detection and characterization of lung nodules. A detection sensitivity of 81.4% and an overall texture classification sensitivity of 77.2% were measured utilizing CAD. CAD has the potential to increase single reader detection rate, however, improvement in texture classification is required.

[Studies and Real-World Experience Regarding the Clinical Application of Artificial Intelligence Software for Lung Nodule Detection]

This article discusses studies and real-world experiences related to the clinical application of artificial intelligence-based computer-aided detection (AI-CAD) software (LuCAS-plus, Monitor Corporation) in detecting pulmonary nodules. During clinical trials for lung cancer screening, AI-CAD exhibited performance comparable to that of medical professionals in terms of sensitivity and specificity. Studies revealed that applying AI-CAD for diagnosing pulmonary metastases led to high detection rates. The use of a nodule matching algorithm in diagnosing pulmonary metastases significantly reduced false non-metastasis results. In clinical settings, implementing AI-CAD enhanced the efficiency of pulmonary nodule detection, saving time and effort during CT reading. Overall, AI-CAD is expected to offer substantial support for lung cancer screening and the interpretation of chest CT scans for malignant tumor surveillance.

Influence of CT dose reduction on AI-driven malignancy estimation of incidental pulmonary nodules

OBJECTIVES

The purpose of this study was to determine the influence of dose reduction on a commercially available lung cancer prediction convolutional neuronal network (LCP-CNN).

METHODS

CT scans from a cohort provided by the local lung cancer center (n = 218) with confirmed pulmonary malignancies and their corresponding reduced dose simulations (25% and 5% dose) were subjected to the LCP-CNN. The resulting LCP scores (scale 1-10, increasing malignancy risk) and the proportion of correctly classified nodules were compared. The cohort was divided into a low-, medium-, and high-risk group based on the respective LCP scores; shifts between the groups were studied to evaluate the potential impact on nodule management. Two different malignancy risk score thresholds were analyzed: a higher threshold of ≥ 9 ("rule-in" approach) and a lower threshold of > 4 ("rule-out" approach).

RESULTS

In total, 169 patients with 196 nodules could be included (mean age ± SD, 64.5 ± 9.2 year; 49% females). Mean LCP scores for original, 25% and 5% dose levels were 8.5 ± 1.7, 8.4 ± 1.7 (p > 0.05 vs. original dose) and 8.2 ± 1.9 (p < 0.05 vs. original dose), respectively. The proportion of correctly classified nodules with the "rule-in" approach decreased with simulated dose reduction from 58.2 to 56.1% (p = 0.34) and to 52.0% for the respective dose levels (p = 0.01). For the "rule-out" approach the respective values were 95.9%, 96.4%, and 94.4% (p = 0.12). When reducing the original dose to 25%/5%, eight/twenty-two nodules shifted to a lower, five/seven nodules to a higher malignancy risk group.

CONCLUSION

CT dose reduction may affect the analyzed LCP-CNN regarding the classification of pulmonary malignancies and potentially alter pulmonary nodule management.

CLINICAL RELEVANCE STATEMENT

Utilization of a "rule-out" approach with a lower malignancy risk threshold prevents underestimation of the nodule malignancy risk for the analyzed software, especially in high-risk cohorts.

KEY POINTS

• LCP-CNN may be affected by CT image parameters such as noise resulting from low-dose CT acquisitions. • CT dose reduction can alter pulmonary nodule management recommendations by affecting the outcome of the LCP-CNN. • Utilization of a lower malignancy risk threshold prevents underestimation of pulmonary malignancies in high-risk cohorts.

Lung Cancer Surgery in Octogenarians: Implications and Advantages of Artificial Intelligence in the Preoperative Assessment

The general world population is aging and patients are often diagnosed with early-stage lung cancer at an advanced age. Several studies have shown that age is not itself a contraindication for lung cancer surgery, and therefore, more and more octogenarians with early-stage lung cancer are undergoing surgery with curative intent. However, octogenarians present some peculiarities that make surgical treatment more challenging, so an accurate preoperative selection is mandatory. In recent years, new artificial intelligence techniques have spread worldwide in the diagnosis, treatment, and therapy of lung cancer, with increasing clinical applications. However, there is still no evidence coming out from trials specifically designed to assess the potential of artificial intelligence in the preoperative evaluation of octogenarian patients. The aim of this narrative review is to investigate, through the analysis of the available international literature, the advantages and implications that these tools may have in the preoperative assessment of this particular category of frail patients. In fact, these tools could represent an important support in the decision-making process, especially in octogenarian patients in whom the diagnostic and therapeutic options are often questionable. However, these technologies are still developing, and a strict human-led process is mandatory.

Usefulness of longitudinal nodule-matching algorithm in computer-aided diagnosis of new pulmonary metastases on cancer surveillance CT scans

Background

Detecting new pulmonary metastases by comparing serial computed tomography (CT) scans is crucial, but a repetitive and time-consuming task that burdens the radiologists' workload. This study aimed to evaluate the usefulness of a nodule-matching algorithm with deep learning-based computer-aided detection (DL-CAD) in diagnosing new pulmonary metastases on cancer surveillance CT scans.

Methods

Among patients who underwent pulmonary metastasectomy between 2014 and 2018, 65 new pulmonary metastases missed by interpreting radiologists on cancer surveillance CT (Time 2) were identified after a retrospective comparison with the previous CT (Time 1). First, DL-CAD detected nodules in Time 1 and Time 2 CT images. All nodules detected at Time 2 were initially considered metastasis candidates. Second, the nodule-matching algorithm was used to assess the correlation between the nodules from the two CT scans and to classify the nodules at Time 2 as "new" or "pre-existing". Pre-existing nodules were excluded from metastasis candidates. We evaluated the performance of DL-CAD with the nodule-matching algorithm, based on its sensitivity, false-metastasis candidates per scan, and positive predictive value (PPV).

Results

A total of 475 lesions were detected by DL-CAD at Time 2. Following a radiologist review, the lesions were categorized as metastases (n=54), benign nodules (n=392), and non-nodules (n=29). Upon comparison of nodules at Time 1 and 2 using the nodule-matching algorithm, all metastases were classified as new nodules without any matching errors. Out of 421 benign lesions, 202 (48.0%) were identified as pre-existing and subsequently excluded from the pool of metastasis candidates through the nodule-matching algorithm. As a result, false-metastasis candidates per CT scan decreased by 47.9% (from 7.1 to 3.7, P<0.001) and the PPV increased from 11.4% to 19.8% (P<0.001), while maintaining sensitivity.

Conclusions

The nodule-matching algorithm improves the diagnostic performance of DL-CAD for new pulmonary metastases, by lowering the number of false-metastasis candidates without compromising sensitivity.

An artificial intelligence-assisted diagnostic system for the prediction of benignity and malignancy of pulmonary nodules and its practical value for patients with different clinical characteristics

Objectives

This study aimed to explore the value of an artificial intelligence (AI)-assisted diagnostic system in the prediction of pulmonary nodules.

Methods

The AI system was able to make predictions of benign or malignant nodules. 260 cases of solitary pulmonary nodules (SPNs) were divided into 173 malignant cases and 87 benign cases based on the surgical pathological diagnosis. A stratified data analysis was applied to compare the diagnostic effectiveness of the AI system to distinguish between the subgroups with different clinical characteristics.

Results

The accuracy of AI system in judging benignity and malignancy of the nodules was 75.77% (p < 0.05). We created an ROC curve by calculating the true positive rate (TPR) and the false positive rate (FPR) at different threshold values, and the AUC was 0.755. Results of the stratified analysis were as follows. (1) By nodule position: the AUC was 0.677, 0.758, 0.744, 0.982, and 0.725, respectively, for the nodules in the left upper lobe, left lower lobe, right upper lobe, right middle lobe, and right lower lobe. (2) By nodule size: the AUC was 0.778, 0.771, and 0.686, respectively, for the nodules measuring 5-10, 10-20, and 20-30 mm in diameter. (3) The predictive accuracy was higher for the subsolid pulmonary nodules than for the solid ones (80.54 vs. 66.67%).

Conclusion

The AI system can be applied to assist in the prediction of benign and malignant pulmonary nodules. It can provide a valuable reference, especially for the diagnosis of subsolid nodules and small nodules measuring 5-10 mm in diameter.

A Synthesizing Semantic Characteristics Lung Nodules Classification Method Based on 3D Convolutional Neural Network

Early detection is crucial for the survival and recovery of lung cancer patients. Computer-aided diagnosis system can assist in the early diagnosis of lung cancer by providing decision support. While deep learning methods are increasingly being applied to tasks such as CAD (Computer-aided diagnosis system), these models lack interpretability. In this paper, we propose a convolutional neural network model that combines semantic characteristics (SCCNN) to predict whether a given pulmonary nodule is malignant. The model synthesizes the advantages of multi-view, multi-task and attention modules in order to fully simulate the actual diagnostic process of radiologists. The 3D (three dimensional) multi-view samples of lung nodules are extracted by spatial sampling method. Meanwhile, semantic characteristics commonly used in radiology reports are used as an auxiliary task and serve to explain how the model interprets. The introduction of the attention module in the feature fusion stage improves the classification of lung nodules as benign or malignant. Our experimental results using the LIDC-IDRI (Lung Image Database Consortium and Image Database Resource Initiative) show that this study achieves 95.45% accuracy and 97.26% ROC (Receiver Operating Characteristic) curve area. The results show that the method we proposed not only realize the classification of benign and malignant compared to standard 3D CNN approaches but can also be used to intuitively explain how the model makes predictions, which can assist clinical diagnosis.

A Narrative Review of the Use of Artificial Intelligence in Breast, Lung, and Prostate Cancer

Artificial intelligence (AI) has been an important topic within radiology. Currently, AI is used clinically to assist with the detection of lesions through detection systems. However, a number of recent studies have demonstrated the increased value of neural networks in radiology. With an increasing number of screening requirements for cancers, this review aims to study the accuracy of the numerous AI models used in the detection and diagnosis of breast, lung, and prostate cancers. This study summarizes pertinent findings from reviewed articles and provides analysis on the relevancy to clinical radiology. This study found that whereas AI is showing continual improvement in radiology, AI alone does not surpass the effectiveness of a radiologist. Additionally, it was found that there are multiple variations on how AI should be integrated with a radiologist's workflow.

Application value of a computer-aided diagnosis and management system for the detection of lung nodules

Background

Computer-aided diagnosis (CAD) systems can help reduce radiologists' workload. This study assessed the value of a CAD system for the detection of lung nodules on chest computed tomography (CT) images.

Methods

The study retrospectively analyzed the CT images of patients who underwent routine health checkups between August 2019 and November 2019 at 3 hospitals in China. All images were first assessed by 2 radiologists manually in a blinded manner, which was followed by assessment with the CAD system. The location and classification of the lung nodules were determined. The final diagnosis was made by a panel of experts, including 2 associate chief radiologists and 1 chief radiologist at the radiology department. The sensitivity for nodule detection and false-positive nodules per case were calculated.

Results

A total of 1,002 CT images were included in the study, and the process was completed for 999 images. The sensitivity of the CAD system and manual detection was 90.19% and 49.88% (P<0.001), respectively. Similar sensitivity was observed between manual detection and the CAD system in lung nodules >15 mm (P=0.08). The false-positive nodules per case for the CAD system were 0.30±0.84 and those for manual detection were 0.24±0.68 (P=0.12). The sensitivity of the CAD system was higher than that of the radiologists, but the increase in the false-positive rate was only slight.

Conclusions

In addition to reducing the workload for medical professionals, a CAD system developed using a deep-learning model was highly effective and accurate in detecting lung nodules and did not demonstrate a meaningfully higher the false-positive rate.

A Review of Deep Learning Techniques for Lung Cancer Screening and Diagnosis Based on CT Images

One of the most common and deadly diseases in the world is lung cancer. Only early identification of lung cancer can increase a patient's probability of survival. A frequently used modality for the screening and diagnosis of lung cancer is computed tomography (CT) imaging, which provides a detailed scan of the lung. In line with the advancement of computer-assisted systems, deep learning techniques have been extensively explored to help in interpreting the CT images for lung cancer identification. Hence, the goal of this review is to provide a detailed review of the deep learning techniques that were developed for screening and diagnosing lung cancer. This review covers an overview of deep learning (DL) techniques, the suggested DL techniques for lung cancer applications, and the novelties of the reviewed methods. This review focuses on two main methodologies of deep learning in screening and diagnosing lung cancer, which are classification and segmentation methodologies. The advantages and shortcomings of current deep learning models will also be discussed. The resultant analysis demonstrates that there is a significant potential for deep learning methods to provide precise and effective computer-assisted lung cancer screening and diagnosis using CT scans. At the end of this review, a list of potential future works regarding improving the application of deep learning is provided to spearhead the advancement of computer-assisted lung cancer diagnosis systems.

"Will I change nodule management recommendations if I change my CAD system?"-impact of volumetric deviation between different CAD systems on lesion management

OBJECTIVES

To evaluate and compare the measurement accuracy of two different computer-aided diagnosis (CAD) systems regarding artificial pulmonary nodules and assess the clinical impact of volumetric inaccuracies in a phantom study.

METHODS

In this phantom study, 59 different phantom arrangements with 326 artificial nodules (178 solid, 148 ground-glass) were scanned at 80 kV, 100 kV, and 120 kV. Four different nodule diameters were used: 5 mm, 8 mm, 10 mm, and 12 mm. Scans were analyzed by a deep-learning (DL)-based CAD and a standard CAD system. Relative volumetric errors (RVE) of each system vs. ground truth and the relative volume difference (RVD) DL-based vs. standard CAD were calculated. The Bland-Altman method was used to define the limits of agreement (LOA). The hypothetical impact on LungRADS classification was assessed for both systems.

RESULTS

There was no difference between the three voltage groups regarding nodule volumetry. Regarding the solid nodules, the RVE of the 5-mm-, 8-mm-, 10-mm-, and 12-mm-size groups for the DL CAD/standard CAD were 12.2/2.8%, 1.3/ - 2.8%, - 3.6/1.5%, and - 12.2/ - 0.3%, respectively. The corresponding values for the ground-glass nodules (GGN) were 25.6%/81.0%, 9.0%/28.0%, 7.6/20.6%, and 6.8/21.2%. The mean RVD for solid nodules/GGN was 1.3/ - 15.2%. Regarding the LungRADS classification, 88.5% and 79.8% of all solid nodules were correctly assigned by the DL CAD and the standard CAD, respectively. 14.9% of the nodules were assigned differently between the systems.

CONCLUSIONS

Patient management may be affected by the volumetric inaccuracy of the CAD systems and hence demands supervision and/or manual correction by a radiologist.

KEY POINTS

• The DL-based CAD system was more accurate in the volumetry of GGN and less accurate regarding solid nodules than the standard CAD system. • Nodule size and attenuation have an effect on the measurement accuracy of both systems; tube voltage has no effect on measurement accuracy. • Measurement inaccuracies of CAD systems can have an impact on patient management, which demands supervision by radiologists.

Qureshi A, Li J, Mahmood T. On the Analyses of Medical Images Using Traditional Machine Learning Techniques and Convolutional Neural Networks

Convolutional neural network (CNN) has shown dissuasive accomplishment on different areas especially Object Detection, Segmentation, Reconstruction (2D and 3D), Information Retrieval, Medical Image Registration, Multi-lingual translation, Local language Processing, Anomaly Detection on video and Speech Recognition. CNN is a special type of Neural Network, which has compelling and effective learning ability to learn features at several steps during augmentation of the data. Recently, different interesting and inspiring ideas of Deep Learning (DL) such as different activation functions, hyperparameter optimization, regularization, momentum and loss functions has improved the performance, operation and execution of CNN Different internal architecture innovation of CNN and different representational style of CNN has significantly improved the performance. This survey focuses on internal taxonomy of deep learning, different models of vonvolutional neural network, especially depth and width of models and in addition CNN components, applications and current challenges of deep learning.

Diagnostic Performance of Artificial Intelligence-Based Computer-Aided Detection and Diagnosis in Pediatric Radiology: A Systematic Review

Artificial intelligence (AI)-based computer-aided detection and diagnosis (CAD) is an important research area in radiology. However, only two narrative reviews about general uses of AI in pediatric radiology and AI-based CAD in pediatric chest imaging have been published yet. The purpose of this systematic review is to investigate the AI-based CAD applications in pediatric radiology, their diagnostic performances and methods for their performance evaluation. A literature search with the use of electronic databases was conducted on 11 January 2023. Twenty-three articles that met the selection criteria were included. This review shows that the AI-based CAD could be applied in pediatric brain, respiratory, musculoskeletal, urologic and cardiac imaging, and especially for pneumonia detection. Most of the studies (93.3%, 14/15; 77.8%, 14/18; 73.3%, 11/15; 80.0%, 8/10; 66.6%, 2/3; 84.2%, 16/19; 80.0%, 8/10) reported model performances of at least 0.83 (area under receiver operating characteristic curve), 0.84 (sensitivity), 0.80 (specificity), 0.89 (positive predictive value), 0.63 (negative predictive value), 0.87 (accuracy), and 0.82 (F1 score), respectively. However, a range of methodological weaknesses (especially a lack of model external validation) are found in the included studies. In the future, more AI-based CAD studies in pediatric radiology with robust methodology should be conducted for convincing clinical centers to adopt CAD and realizing its benefits in a wider context.

Prediction of histologic types in solid lung lesions using preoperative contrast-enhanced CT

OBJECTIVES

This study aimed to develop and validate a predicting model for the histologic classification of solid lung lesions based on preoperative contrast-enhanced CT.

METHODS

A primary dataset of 1012 patients from Tianjin Medical University Cancer Institute and Hospital (TMUCIH) was randomly divided into a development cohort (708) and an internal validation cohort (304). Patients from the Second Hospital of Shanxi Medical University (SHSMU) were set as an external validation cohort (212). Two clinical factors (age, gender) and twenty-one characteristics on contrast-enhanced CT were used to construct a multinomial multivariable logistic regression model for the classification of seven common histologic types of solid lung lesions. The area under the receiver operating characteristic curve was used to assess the diagnostic performance of the model in the development and validation cohorts, separately.

RESULTS

Multivariable analysis showed that two clinical factors and twenty-one characteristics on contrast-enhanced CT were predictive in lung lesion histologic classification. The mean AUC of the proposed model for histologic classification was 0.95, 0.94, and 0.92 in the development, internal validation, and external validation cohort, respectively. When determining the malignancy of lung lesions based on histologic types, the mean AUC of the model was 0.88, 0.86, and 0.90 in three cohorts.

CONCLUSIONS

We demonstrated that by utilizing both clinical and CT characteristics on contrast-enhanced CT images, the proposed model could not only effectively stratify histologic types of solid lung lesions, but also enabled accurate assessment of lung lesion malignancy. Such a model has the potential to avoid unnecessary surgery for patients and to guide clinical decision-making for preoperative treatment.

KEY POINTS

• Clinical and CT characteristics on contrast-enhanced CT could be used to differentiate histologic types of solid lung lesions. • Predicting models using preoperative contrast-enhanced CT could accurately assessment of tumor malignancy based on predicted histologic types.

Artificial intelligence in lung cancer: current applications and perspectives

Artificial intelligence (AI) has been a very active research topic over the last years and thoracic imaging has particularly benefited from the development of AI and in particular deep learning. We have now entered a phase of adopting AI into clinical practice. The objective of this article was to review the current applications and perspectives of AI in thoracic oncology. For pulmonary nodule detection, computer-aided detection (CADe) tools have been commercially available since the early 2000s. The more recent rise of deep learning and the availability of large annotated lung nodule datasets have allowed the development of new CADe tools with fewer false-positive results per examination. Classical machine learning and deep-learning methods were also used for pulmonary nodule segmentation allowing nodule volumetry and pulmonary nodule characterization. For pulmonary nodule characterization, radiomics and deep-learning approaches were used. Data from the National Lung Cancer Screening Trial (NLST) allowed the development of several computer-aided diagnostic (CADx) tools for diagnosing lung cancer on chest computed tomography. Finally, AI has been used as a means to perform virtual biopsies and to predict response to treatment or survival. Thus, many detection, characterization and stratification tools have been proposed, some of which are commercially available.

Artificial intelligence: A critical review of applications for lung nodule and lung cancer

Artificial intelligence (AI) is a broad concept that usually refers to computer programs that can learn from data and perform certain specific tasks. In the recent years, the growth of deep learning, a successful technique for computer vision tasks that does not require explicit programming, coupled with the availability of large imaging databases fostered the development of multiple applications in the medical imaging field, especially for lung nodules and lung cancer, mostly through convolutional neural networks (CNN). Some of the first applications of AI is this field were dedicated to automated detection of lung nodules on X-ray and computed tomography (CT) examinations, with performances now reaching or exceeding those of radiologists. For lung nodule segmentation, CNN-based algorithms applied to CT images show excellent spatial overlap index with manual segmentation, even for irregular and ground glass nodules. A third application of AI is the classification of lung nodules between malignant and benign, which could limit the number of follow-up CT examinations for less suspicious lesions. Several algorithms have demonstrated excellent capabilities for the prediction of the malignancy risk when a nodule is discovered. These different applications of AI for lung nodules are particularly appealing in the context of lung cancer screening. In the field of lung cancer, AI tools applied to lung imaging have been investigated for distinct aims. First, they could play a role for the non-invasive characterization of tumors, especially for histological subtype and somatic mutation predictions, with a potential therapeutic impact. Additionally, they could help predict the patient prognosis, in combination to clinical data. Despite these encouraging perspectives, clinical implementation of AI tools is only beginning because of the lack of generalizability of published studies, of an inner obscure working and because of limited data about the impact of such tools on the radiologists' decision and on the patient outcome. Radiologists must be active participants in the process of evaluating AI tools, as such tools could support their daily work and offer them more time for high added value tasks.

Registered Clinical Trials for Artificial Intelligence in Lung Disease: A Scoping Review on ClinicalTrials.gov

Clinical trials are the most effective tools to evaluate the advantages of various diagnostic and treatment modalities. AI used in medical issues, including screening, diagnosis, and treatment decisions, improves health outcomes and patient experiences. This study's objective was to investigate the traits of registered trials on artificial intelligence for lung disease. Clinical studies on AI for lung disease that were present in the ClinicalTrials.gov database were searched, and fifty-three registered trials were included. Forty-six (72.1%) were observational trials, compared to seven (27.9%) that were interventional trials. Only eight trials (15.4%) were completed. Thirty (56.6%) trials were accepting applicants. Clinical studies often included a large number of cases; for example, 24 (32.0%) trials included samples of 100-1000 cases, while 14 (17.5%) trials included samples of 1000-2000 cases. Of the interventional trials, twenty (15.7%) were retrospective studies and twenty (65.7%) were prospective studies.

Artificial Intellgence in the Era of Precision Oncological Imaging

Rapid-paced development and adaptability of artificial intelligence algorithms have secured their almost ubiquitous presence in the field of oncological imaging. Artificial intelligence models have been created for a variety of tasks, including risk stratification, automated detection, and segmentation of lesions, characterization, grading and staging, prediction of prognosis, and treatment response. Soon, artificial intelligence could become an essential part of every step of oncological workup and patient management. Integration of neural networks and deep learning into radiological artificial intelligence algorithms allow for extrapolating imaging features otherwise inaccessible to human operators and pave the way to truly personalized management of oncological patients.Although a significant proportion of currently available artificial intelligence solutions belong to basic and translational cancer imaging research, their progressive transfer to clinical routine is imminent, contributing to the development of a personalized approach in oncology. We thereby review the main applications of artificial intelligence in oncological imaging, describe the example of their successful integration into research and clinical practice, and highlight the challenges and future perspectives that will shape the field of oncological radiology.

Development and performance evaluation of a deep learning lung nodule detection system

BACKGROUND

Lung cancer is the leading cause of cancer-related deaths throughout the world. Chest computed tomography (CT) is now widely used in the screening and diagnosis of lung cancer due to its effectiveness. Radiologists must identify each small nodule shadow from 3D volume images, which is very burdensome and often results in missed nodules. To address these challenges, we developed a computer-aided detection (CAD) system that automatically detects lung nodules in CT images.

METHODS

A total of 1997 chest CT scans were collected for algorithm development. The algorithm was designed using deep learning technology. In addition to evaluating detection performance on various public datasets, its robustness to changes in radiation dose was assessed by a phantom study. To investigate the clinical usefulness of the CAD system, a reader study was conducted with 10 doctors, including inexperienced and expert readers. This study investigated whether the use of the CAD as a second reader could prevent nodular lesions in lungs that require follow-up examinations from being overlooked. Analysis was performed using the Jackknife Free-Response Receiver-Operating Characteristic (JAFROC).

RESULTS

The CAD system achieved sensitivity of 0.98/0.96 at 3.1/7.25 false positives per case on two public datasets. Sensitivity did not change within the range of practical doses for a study using a phantom. A second reader study showed that the use of this system significantly improved the detection ability of nodules that could be picked up clinically (p = 0.026).

CONCLUSIONS

We developed a deep learning-based CAD system that is robust to imaging conditions. Using this system as a second reader increased detection performance.

Artificial Intelligence (AI) for Lung Nodules, From the AJR Special Series on AI Applications

Interest in artificial intelligence (AI) applications for lung nodules continues to grow among radiologists, particularly with the expanding eligibility criteria and clinical utilization of lung cancer screening CT. AI has been heavily investigated for detecting and characterizing lung nodules and for guiding prognostic assessment. AI tools have also been used for image postprocessing (e.g., rib suppression on radiography or vessel suppression on CT) and for noninterpretive aspects of reporting and workflow, including management of nodule follow-up. Despite growing interest in and rapid development of AI tools and FDA approval of AI tools for pulmonary nodule evaluation, integration into clinical practice has been limited. Challenges to clinical adoption have included concerns about generalizability, regulatory issues, technical hurdles in implementation, and human skepticism. Further validation of AI tools for clinical use and demonstration of benefit in terms of patient-oriented outcomes also are needed. This article provides an overview of potential applications of AI tools in the imaging evaluation of lung nodules and discusses the challenges faced by practices interested in clinical implementation of such tools.

Development of Deep Learning-based Automatic Scan Range Setting Model for Lung Cancer Screening Low-dose CT Imaging

RATIONALE AND OBJECTIVES

To develop an automatic setting of a deep learning-based system for detecting low-dose computed tomography (CT) lung cancer screening scan range and compare its efficiency with the radiographer's performance.

MATERIALS AND METHODS

This retrospective study was performed using 1984 lung cancer screening low-dose CT scans obtained between November 2019 and May 2020. Among 1984 CT scans, 600 CT scans were considered suitable for an observational study to explore the relationship between the scout landmarks and the actual lung boundaries. Further, 1144 CT scans data set was used for the development of a deep learning-based algorithm. This data set was split into an 8:2 ratio divided into a training set (80%, n = 915) and a validation set (20%, n = 229). The performance of the deep learning algorithm was evaluated in the test set (n = 240) using actual lung boundaries and radiographers' scan ranges.

RESULTS

The mean differences between the upper and lower boundaries of the deep learning-based algorithm and the actual lung boundaries were 4.72 ± 3.15 mm and 16.50 ± 14.06 mm, respectively. The accuracy and over-scanning of the scan ranges generated by the system were 97.08% (233/240) and 0% (0/240) for the upper boundary, and 96.25% (231/240) and 29.58% (71/240) for the lower boundary.

CONCLUSION

The developed deep learning-based algorithm system can effectively predict lung cancer screening low-dose CT scan range with high accuracy using only the frontal scout.

A narrative review of deep learning applications in lung cancer research: from screening to prognostication

Background and Objective

Deep learning (DL) algorithms have been developed for various tasks, including lung nodule detection on chest radiographs or lung cancer computed tomography screening, potential candidate selection in lung cancer screening, malignancy prediction for indeterminate pulmonary nodules, lung cancer staging, treatment response prediction, prognostication, and prediction of genetic mutations in lung cancer. Furthermore, these DL algorithms have been applied in various clinical settings in order for them to be generalized in real-world clinical practice. Multiple DL algorithms have been corroborated to be on par with experts or current clinical prediction models for several specific tasks. However, no article has yet comprehensively reviewed DL algorithms dedicated to lung cancer research. This narrative review presents an overview of the literature dealing with DL techniques applied in lung cancer research and briefly summarizes the results according to the DL algorithms’ clinical use cases.

Methods

we performed a narrative review by searching the Embase and OVID-MEDLINE databases for articles published in English from October, 2016 until September, 2021 and reviewing the bibliographies of key references to identify important literature related to DL in lung cancer research. The background, development, results, and clinical implications of each DL algorithm are briefly discussed. Lastly, we end this review article by highlighting future directions in lung cancer research using DL techniques.

Key Content and Findings

DL algorithms have been introduced to show comparable or higher performance than human experts in various clinical settings. Specifically, they have been actively applied to detect lung nodules in chest radiographs or computed tomography (CT) examinations, optimize candidate selection for lung cancer screening (LCS), predict the malignancy of lung nodules, stage lung cancer, and predict treatment response, patients’ prognoses, and genetic mutations in lung cancers.

Conclusions

DL algorithms have corroborated their potential value for various tasks, ranging from lung cancer screening to prognostication of lung cancer patients. Future research is warranted for the clinical application of these algorithms in daily clinical practice and verification of their real-world clinical usefulness.

Diagnostic validation of a deep learning nodule detection algorithm in low-dose chest CT: determination of optimized dose thresholds in a virtual screening scenario

OBJECTIVES

This study was conducted to evaluate the effect of dose reduction on the performance of a deep learning (DL)-based computer-aided diagnosis (CAD) system regarding pulmonary nodule detection in a virtual screening scenario.

METHODS

Sixty-eight anthropomorphic chest phantoms were equipped with 329 nodules (150 ground glass, 179 solid) with four sizes (5 mm, 8 mm, 10 mm, 12 mm) and scanned with nine tube voltage/current combinations. The examinations were analyzed by a commercially available DL-based CAD system. The results were compared by a comparison of proportions. Logistic regression was performed to evaluate the impact of tube voltage, tube current, nodule size, nodule density, and nodule location.

RESULTS

The combination with the lowest effective dose (E) and unimpaired detection rate was 80 kV/50 mAs (sensitivity: 97.9%, mean false-positive rate (FPR): 1.9, mean CTDIvol: 1.2 ± 0.4 mGy, mean E: 0.66 mSv). Logistic regression revealed that tube voltage and current had the greatest impact on the detection rate, while nodule size and density had no significant influence.

CONCLUSIONS

The optimal tube voltage/current combination proposed in this study (80 kV/50 mAs) is comparable to the proposed combinations in similar studies, which mostly dealt with conventional CAD software. Modification of tube voltage and tube current has a significant impact on the performance of DL-based CAD software in pulmonary nodule detection regardless of their size and composition.

KEY POINTS

• Modification of tube voltage and tube current has a significant impact on the performance of deep learning-based CAD software. • Nodule size and composition have no significant impact on the software's performance. • The optimal tube voltage/current combination for the examined software is 80 kV/50 mAs.

Computer-aided detection of pulmonary nodules based on convolutional neural networks: a review

Computer-aided detection (CADe) technology has been proven to increase the detection rate of pulmonary nodules that has important clinical significance for the early diagnosis of lung cancer. In this study, we systematically review the latest techniques in pulmonary nodule CADe based on deep learning models with convolutional neural networks in computed tomography images. First, the brief descriptions and popular architecture of convolutional neural networks are introduced. Second, several common public databases and evaluation metrics are briefly described. Third, state-of-the-art approaches with excellent performances are selected. Subsequently, we combine the clinical diagnostic process and the traditional four steps of pulmonary nodule CADe into two stages, namely, data preprocessing and image analysis. Further, the major optimizations of deep learning models and algorithms are highlighted according to the progressive evaluation effect of each method, and some clinical evidence is added. Finally, various methods are summarized and compared. The innovative or valuable contributions of each method are expected to guide future research directions. The analyzed results show that deep learning-based methods significantly transformed the detection of pulmonary nodules, and the design of these methods can be inspired by clinical imaging diagnostic procedures. Moreover, focusing on the image analysis stage will result in improved returns. In particular, optimal results can be achieved by optimizing the steps of candidate nodule generation and false positive reduction. End-to-end methods, with greater operating speeds and lower computational consumptions, are superior to other methods in CADe of pulmonary nodules.

Performance of a deep learning-based lung nodule detection system as an alternative reader in a Chinese lung cancer screening program

OBJECTIVE

To evaluate the performance of a deep learning-based computer-aided detection (DL-CAD) system in a Chinese low-dose CT (LDCT) lung cancer screening program.

MATERIALS AND METHODS

One-hundred-and-eighty individuals with a lung nodule on their baseline LDCT lung cancer screening scan were randomly mixed with screenees without nodules in a 1:1 ratio (total: 360 individuals). All scans were assessed by double reading and subsequently processed by an academic DL-CAD system. The findings of double reading and the DL-CAD system were then evaluated by two senior radiologists to derive the reference standard. The detection performance was evaluated by the Free Response Operating Characteristic curve, sensitivity and false-positive (FP) rate. The senior radiologists categorized nodules according to nodule diameter, type (solid, part-solid, non-solid) and Lung-RADS.

RESULTS

The reference standard consisted of 262 nodules ≥ 4 mm in 196 individuals; 359 findings were considered false positives. The DL-CAD system achieved a sensitivity of 90.1% with 1.0 FP/scan for detection of lung nodules regardless of size or type, whereas double reading had a sensitivity of 76.0% with 0.04 FP/scan (P = 0.001). The sensitivity for detection of nodules ≥ 4 - ≤ 6 mm was significantly higher with DL-CAD than with double reading (86.3% vs. 58.9% respectively; P = 0.001). Sixty-three nodules were only identified by the DL-CAD system, and 27 nodules only found by double reading. The DL-CAD system reached similar performance compared to double reading in Lung-RADS 3 (94.3% vs. 90.0%, P = 0.549) and Lung-RADS 4 nodules (100.0% vs. 97.0%, P = 1.000), but showed a higher sensitivity in Lung-RADS 2 (86.2% vs. 65.4%, P < 0.001).

CONCLUSIONS

The DL-CAD system can accurately detect pulmonary nodules on LDCT, with an acceptable false-positive rate of 1 nodule per scan and has higher detection performance than double reading. This DL-CAD system may assist radiologists in nodule detection in LDCT lung cancer screening.

Are We Overdoing It? Changes in Diagnostic Imaging Workload during the Years 2010-2020 including the Impact of the SARS-CoV-2 Pandemic

Since the 1990s, there has been a significant increase in the number of imaging examinations as well as a related increase in the healthcare expenditure and the exposure of the population to X-rays. This study aimed to analyze the workload trends in radiology during the last decade, including the impact of COVID-19 in a single university hospital in Poland and to identify possible solutions to the challenges that radiology could face in the future. We compared the annual amount of computed tomography (CT), radiography (X-ray), and ultrasound (US) examinations performed between the years 2010 and 2020 and analyzed the changes in the number of practicing radiologists in Poland. The mean number of patients treated in our hospital was 60,727 per year. During the last decade, the number of CT and US examinations nearly doubled (from 87.4 to 155.7 and from 52.1 to 86.5 per 1000 patients in 2010 and 2020 respectively), while X-ray examinations decreased from 115.1 to 96.9 per 1000 patients. The SARS-CoV-2 pandemic did not change the workload trends as more chest examinations were performed. AI, which contributed to the COVID-19 diagnosis, could aid radiologists in the future with the growing workload by increasing the efficiency of radiology departments as well as by potentially minimizing the related costs.

A Robust Deep Learning Segmentation Method for Hematoma Volumetric Detection in Intracerebral Hemorrhage

BACKGROUND AND PURPOSE

Hematoma volume (HV) is a significant diagnosis for determining the clinical stage and therapeutic approach for intracerebral hemorrhage (ICH). The aim of this study is to develop a robust deep learning segmentation method for the fast and accurate HV analysis using computed tomography.

METHODS

A novel dimension reduction UNet (DR-UNet) model was developed for computed tomography image segmentation and HV measurement. Two data sets, 512 ICH patients with 12 568 computed tomography slices in the retrospective data set and 50 ICH patients with 1257 slices in the prospective data set, were used for network training, validation, and internal and external testing. Moreover, 13 irregular hematoma cases, 11 subdural and epidural hematoma cases, and 50 different HV cases into 3 groups (<30, 30-60, and >60 mL) were selected to further evaluate the robustness of DR-UNet. The image segmentation performance of DR-UNet was compared with those of UNet, the fuzzy clustering method, and the active contour method. The HV measurement performance was compared using DR-UNet, UNet, and the Coniglobus formula method.

RESULTS

Using DR-UNet, the segmentation model achieved a performance similar to that of expert clinicians in 2 independent test data sets containing internal testing data (Dice of 0.861±0.139) and external testing data (Dice of 0.874±0.130). The HV measurement derived from DR-UNet was strongly correlated with that from manual segmentation (R²=0.9979; P<0.0001). In the irregularly shaped hematoma group and the subdural and epidural hematoma group, DR-UNet was more robust than UNet in both hematoma segmentation and HV measurement. There is no statistical significance in segmentation accuracy among 3 different HV groups.

CONCLUSIONS

DR-UNet can segment hematomas from the computed tomography scans of ICH patients and quantify the HV with better accuracy and greater efficiency than the main existing methods and with similar performance to expert clinicians. Due to robust performance and stable segmentation on different ICHs, DR-UNet could facilitate the development of deep learning systems for a variety of clinical applications.

Application of computer-aided diagnosis for Lung-RADS categorization in CT screening for lung cancer: effect on inter-reader agreement

OBJECTIVES

To evaluate the effects of computer-aided diagnosis (CAD) on inter-reader agreement in Lung Imaging Reporting and Data System (Lung-RADS) categorization.

METHODS

Two hundred baseline CT scans covering all Lung-RADS categories were randomly selected from the National Lung Cancer Screening Trial. Five radiologists independently reviewed the CT scans and assigned Lung-RADS categories without CAD and with CAD. The CAD system presented up to five of the most risk-dominant nodules with measurements and predicted Lung-RADS category. Inter-reader agreement was analyzed using multirater Fleiss κ statistics.

RESULTS

The five readers reported 139-151 negative screening results without CAD and 126-142 with CAD. With CAD, readers tended to upstage (average, 12.3%) rather than downstage Lung-RADS category (average, 4.4%). Inter-reader agreement of five readers for Lung-RADS categorization was moderate (Fleiss kappa, 0.60 [95% confidence interval, 0.57, 0.63]) without CAD, and slightly improved to substantial (Fleiss kappa, 0.65 [95% CI, 0.63, 0.68]) with CAD. The major cause for disagreement was assignment of different risk-dominant nodules in the reading sessions without and with CAD (54.2% [201/371] vs. 63.6% [232/365]). The proportion of disagreement in nodule size measurement was reduced from 5.1% (102/2000) to 3.1% (62/2000) with the use of CAD (p < 0.001). In 31 cancer-positive cases, substantial management discrepancies (category 1/2 vs. 4A/B) between reader pairs decreased with application of CAD (pooled sensitivity, 85.2% vs. 91.6%; p = 0.004).

CONCLUSIONS

Application of CAD demonstrated a minor improvement in inter-reader agreement of Lung-RADS category, while showing the potential to reduce measurement variability and substantial management change in cancer-positive cases.

KEY POINTS

• Inter-reader agreement of five readers for Lung-RADS categorization was minimally improved by application of CAD, with a Fleiss kappa value of 0.60 to 0.65. • The major cause for disagreement was assignment of different risk-dominant nodules in the reading sessions without and with CAD (54.2% vs. 63.6%). • In 31 cancer-positive cases, substantial management discrepancies between reader pairs, referring to a difference in follow-up interval of at least 9 months (category 1/2 vs. 4A/B), were reduced in half by application of CAD (32/310 to 16/310) (pooled sensitivity, 85.2% vs. 91.6%; p = 0.004).

Performance and reading time of lung nodule identification on multidetector CT with or without an artificial intelligence-powered computer-aided detection system

AIM

To compare the performance and reading time of different readers using automatic artificial intelligence (AI)-powered computer-aided detection (CAD) to detect lung nodules in different reading modes.

MATERIALS AND METHODS

One hundred and fifty multidetector computed tomography (CT) datasets containing 340 nodules ≤10 mm in diameter were collected retrospectively. A CAD with vessel-suppressed function was used to interpret the images. Three junior and three senior readers were assigned to read (1) CT images without CAD, (2) second-read using CAD in which CAD was applied only after initial unassisted assessment, and (3) a concurrent read with CAD in which CAD was applied at the start of assessment. Diagnostic performances and reading times were compared using analysis of variance.

RESULTS

For all readers, the mean sensitivity improved from 64% (95% confidence interval [CI]: 62%, 66%) for the without-CAD mode to 82% (95% CI: 80%, 84%) for the second-reading mode and to 80% (95% CI: 79%, 82%) for the concurrent-reading mode (p<0.001). There was no significant difference between the two modes in terms of the mean sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for both junior and senior readers and all readers (p>0.05). The reading time of all readers was significantly shorter for the concurrent-reading mode (124 ± 25 seconds) compared to without CAD (156 ± 34 seconds; p<0.001) and the second-reading mode (197 ± 46 seconds; p<0.001).

CONCLUSION

In CAD for lung nodules at CT, the second-reading mode and concurrent-reading mode may improve detection performance for all readers in both screening and clinical routine practice. Concurrent use of CAD is more efficient for both junior and senior readers.

High precision localization of pulmonary nodules on chest CT utilizing axial slice number labels

BACKGROUND

Reidentification of prior nodules for temporal comparison is an important but time-consuming step in lung cancer screening. We develop and evaluate an automated nodule detector that utilizes the axial-slice number of nodules found in radiology reports to generate high precision nodule predictions.

METHODS

888 CTs from Lung Nodule Analysis were used to train a 2-dimensional (2D) object detection neural network. A pipeline of 2D object detection, 3D unsupervised clustering, false positive reduction, and axial-slice numbers were used to generate nodule candidates. 47 CTs from the National Lung Cancer Screening Trial (NLST) were used for model evaluation.

RESULTS

Our nodule detector achieved a precision of 0.962 at a recall of 0.573 on the NLST test set for any nodule. When adjusting for unintended nodule predictions, we achieved a precision of 0.931 at a recall 0.561, which corresponds to 0.06 false positives per CT. Error analysis revealed better detection of nodules with soft tissue attenuation compared to ground glass and undeterminable attenuation. Nodule margins, size, location, and patient demographics did not differ between correct and incorrect predictions.

CONCLUSIONS

Utilization of axial-slice numbers from radiology reports allowed for development of a lung nodule detector with a low false positive rate compared to prior feature-engineering and machine learning approaches. This high precision nodule detector can reduce time spent on reidentification of prior nodules during lung cancer screening and can rapidly develop new institutional datasets to explore novel applications of computer vision in lung cancer imaging.

Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis

Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.

Can a Novel Deep Neural Network Improve the Computer-Aided Detection of Solid Pulmonary Nodules and the Rate of False-Positive Findings in Comparison to an Established Machine Learning Computer-Aided Detection?

OBJECTIVE

The aim of this study was to compare the performance of 2 approved computer-aided detection (CAD) systems for detection of pulmonary solid nodules (PSNs) in an oncologic cohort. The first CAD system is based on a conventional machine learning approach (VD10F), and the other is based on a deep 3D convolutional neural network (CNN) CAD software (VD20A).

METHODS AND MATERIALS

Nine hundred sixty-seven patients with a total of 2451 PSNs were retrospectively evaluated using the 2 different CAD systems. All patients had thin-slice chest computed tomography (0.6 mm) using 100 kV and 100 mAs and a high-resolution kernel (I50f). The CAD images generated by VD10F were transferred to the PACS for evaluation. The images generated by VD20A were evaluated using a Web browser-based viewer. Finally, a senior radiologist who was blinded for the CAD results examined the thin-slice images of every patient (ground truth).

RESULTS

A total of 2451 PSNs were detected by the senior radiologist. CAD-VD10F detected 1401 true-positive, 143 false-negative, 565 false-positive (FP), and 342 true-negative PSNs, resulting in sensitivity of 90.7%, specificity of 37.7%, positive predictive value of 0.71, and negative predictive value of 0.70. CAD-VD20A detected 1381 true-positive, 163 false-negative, 337 FP, and 570 true-negative PSNs, resulting in sensitivity of 89.4%, specificity of 62.8%, positive predictive value of 0.80, and negative predictive value 0.77, respectively. The rate of FP per scan was 0.6 for CAD-VD10F and 0.3 for CAD-VD20A.

CONCLUSIONS

The new deep learning-based CAD software (VD20A) shows similar sensitivity with the conventional CAD software (VD10F), but a significantly higher specificity.

Artificial Intelligence and its future potential in lung cancer screening

Artificial intelligence (AI) simulates intelligent behavior as well as critical thinking comparable to a human being and can be used to analyze and interpret complex medical data. The application of AI in imaging diagnostics reduces the burden of radiologists and increases the sensitivity of lung cancer screening so that the morbidity and mortality associated with lung cancer can be decreased. In this article, we have tried to evaluate the role of artificial intelligence in lung cancer screening, as well as the future potential and efficiency of AI in the classification of nodules. The relevant studies between 2010-2020 were selected from the PubMed database after excluding animal studies and were analyzed for the contribution of AI. Techniques such as deep learning and machine learning allow automatic characterization and classification of nodules with high precision and promise an advanced lung cancer screening method in the future. Even though several combination models with high performance have been proposed, an effectively validated model for routine use still needs to be improvised. Combining the performance of artificial intelligence with a radiologist's expertise offers a successful outcome with higher accuracy. Thus, we can conclude that higher sensitivity, specificity, and accuracy of lung cancer screening and classification of nodules is possible through the integration of artificial intelligence and radiology. The validation of models and further research is to be carried out to determine the feasibility of this integration.

Sedentary Behavior in Children by Wearable Cameras: Development of an Annotation Protocol

INTRODUCTION

There is increasing evidence that not all types of sedentary behavior have the same harmful effects on children's health. Hence, there has been a growing interest in the use of wearable cameras. The aim of this study is to develop a protocol to categorize children's wearable camera data into sedentary behavior components.

METHODS

Wearable camera data were collected in 3 different samples of children in 2014. A development sample (3 children aged 4-8 years) was used to design the annotation protocol. A training sample (4 children aged 10 years) was used to train 3 different coders. The independent reliability sample (14 children aged 9-11 years) was used for independent coding of wearable camera images and to estimate inter-rater agreement. Data were analyzed in 2018. Cohen's κ was calculated for every rater pair on a per-participant basis. Means and SDs were then calculated across per-participant κ scores.

RESULTS

A total of 41,651 images from 14 participants were considered for analysis. Inter-rater agreement over all raters over all the sedentary behavior components was almost perfect (mean κ=0.85, 95% CI=0.83, 0.87). Inter-rater reliability for screen-based sedentary behavior (mean κ=0.72, 95% CI=0.62, 0.82) and nonscreen sedentary behavior (κ=0.69, 95% CI=0.65, 0.72) showed substantial agreement. Inter-rater reliability for location (κ=0.91, 95% CI=0.88, 0.93) showed almost perfect agreement.

CONCLUSIONS

A reliable annotation protocol to categorize wearable camera data of children into sedentary behavior components was developed. Once applied to larger samples in children, this protocol can ultimately help to better understand the potential harms of screen time and sedentary behavior in children.

Does a Deep Learning-Based Computer-Assisted Diagnosis System Outperform Conventional Double Reading by Radiologists in Distinguishing Benign and Malignant Lung Nodules?

Background

In differentiating indeterminate pulmonary nodules, multiple studies indicated the superiority of deep learning–based computer-assisted diagnosis system (DL-CADx) over conventional double reading by radiologists, which needs external validation. Therefore, our aim was to externally validate the performance of a commercial DL-CADx in differentiating benign and malignant lung nodules.

Methods

In this retrospective study, 233 patients with 261 pathologically confirmed lung nodules were enrolled. Double reading was used to rate each nodule using a four-scale malignancy score system, including unlikely (0–25%), malignancy cannot be completely excluded (25–50%), highly likely (50–75%), and considered as malignant (75–100%), with any disagreement resolved through discussion. DL-CADx automatically rated each nodule with a malignancy likelihood ranging from 0 to 100%, which was then quadrichotomized accordingly. Intraclass correlation coefficient (ICC) was used to evaluate the agreement in malignancy risk rating between DL-CADx and double reading, with ICC value of <0.5, 0.5 to 0.75, 0.75 to 0.9, and >0.9 indicating poor, moderate, good, and perfect agreement, respectively. With malignancy likelihood >50% as cut-off value for malignancy and pathological results as gold standard, sensitivity, specificity, and accuracy were calculated for double reading and DL-CADx, separately.

Results

Among the 261 nodules, 247 nodules were successfully detected by DL-CADx with detection rate of 94.7%. Regarding malignancy rating, DL-CADx was in moderate agreement with double reading (ICC = 0.555, 95% CI 0.424 to 0.655). DL-CADx misdiagnosed 40 true malignant nodules as benign nodules and 30 true benign nodules as malignant nodules with sensitivity, specificity, and accuracy of 79.2, 45.5, and 71.7%, respectively. In contrast, double reading achieved better performance with 16 true malignant nodules misdiagnosed as benign nodules and 26 true benign nodules as malignant nodules with sensitivity, specificity, and accuracy of 91.7, 52.7, and 83.0%, respectively.

Conclusion

Compared with double reading, DL-CADx we used still shows inferior performance in differentiating malignant and benign nodules.

The Use of Artificial Intelligence in the Differentiation of Malignant and Benign Lung Nodules on Computed Tomograms Proven by Surgical Pathology

The purpose of this work was to evaluate the performance of an existing commercially available artificial intelligence (AI) software system in differentiating malignant and benign lung nodules. The AI tool consisted of a vessel-suppression function and a deep-learning-based computer-aided-detection (VS-CAD) analyzer. Fifty patients (32 females, mean age 52 years) with 75 lung nodules (47 malignant and 28 benign) underwent low-dose computed tomography (LDCT) followed by surgical excision and the pathological analysis of their 75 nodules within a 3 month time frame. All 50 cases were then processed by the AI software to generate corresponding VS images and CAD outcomes. All 75 pathologically proven lung nodules were well delineated by vessel-suppressed images. Three (6.4%) of the 47 lung cancer cases, and 11 (39.3%) of the 28 benign nodules were ignored and not detected by the AI without showing a CAD analysis summary. The AI system/radiologists produced a sensitivity and specificity (shown in %) of 93.6/89.4 and 39.3/82.1 in distinguishing malignant from benign nodules, respectively. AI sensitivity was higher than that of radiologists, though not statistically significant (p = 0.712). Specificity obtained by the radiologists was significantly higher than that of the VS-CAD AI (p = 0.003). There was no significant difference between the malignant and benign lesions with respect to age, gender, pure ground-glass pattern, the diameter and location of the nodules, or nodules <6 vs. ≥6 mm. However, more part-solid nodules were proven to be malignant than benign (90.9% vs. 9.1%), and more solid nodules were proven to be benign than malignant (86.7% vs. 13.3%) with statistical significance (p = 0.001 and <0.001, respectively). A larger cohort and prospective study are required to validate the AI performance.

Efficiency of a computer-aided diagnosis (CAD) system with deep learning in detection of pulmonary nodules on 1-mm-thick images of computed tomography

PURPOSE

To evaluate the performance of a deep learning-based computer-aided diagnosis (CAD) system at detecting pulmonary nodules on CT by comparing radiologists' readings with and without CAD.

MATERIALS AND METHODS

A total of 120 chest CT images were randomly selected from patients with suspected lung cancer. The gold standard of nodules ≥ 3 mm was established by a panel of three expert radiologists. Two less experienced radiologists read the images without and afterward with CAD system. Their reading times were recorded.

RESULTS

The radiologists' sensitivity increased from 20.9% to 38.0% with the introduction of CAD. The positive predictive value (PPV) decreased from 70.5% to 61.8%, and the F1-score increased from 32.2% to 47.0%. The sensitivity significantly increased from 13.7% to 32.4% for small nodules (3-6 mm) and from 33.3% to 47.6% for medium nodules (6-10 mm). CAD alone showed a sensitivity of 70.3%, a PPV of 57.9%, and an F1-score of 63.5%. Reading time decreased by 11.3% with the use of CAD.

CONCLUSION

CAD improved the less experienced radiologists' sensitivity in detecting pulmonary nodules of all sizes, especially including a significant improvement in the detection of clinically important-sized medium nodules (6-10 mm) as well as small nodules (3-6 mm) and reduced their reading time.

The Performance of Deep Learning Algorithms on Automatic Pulmonary Nodule Detection and Classification Tested on Different Datasets That Are Not Derived from LIDC-IDRI: A Systematic Review

The aim of this study was to systematically review the performance of deep learning technology in detecting and classifying pulmonary nodules on computed tomography (CT) scans that were not from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) database. Furthermore, we explored the difference in performance when the deep learning technology was applied to test datasets different from the training datasets. Only peer-reviewed, original research articles utilizing deep learning technology were included in this study, and only results from testing on datasets other than the LIDC-IDRI were included. We searched a total of six databases: EMBASE, PubMed, Cochrane Library, the Institute of Electrical and Electronics Engineers, Inc. (IEEE), Scopus, and Web of Science. This resulted in 1782 studies after duplicates were removed, and a total of 26 studies were included in this systematic review. Three studies explored the performance of pulmonary nodule detection only, 16 studies explored the performance of pulmonary nodule classification only, and 7 studies had reports of both pulmonary nodule detection and classification. Three different deep learning architectures were mentioned amongst the included studies: convolutional neural network (CNN), massive training artificial neural network (MTANN), and deep stacked denoising autoencoder extreme learning machine (SDAE-ELM). The studies reached a classification accuracy between 68–99.6% and a detection accuracy between 80.6–94%. Performance of deep learning technology in studies using different test and training datasets was comparable to studies using same type of test and training datasets. In conclusion, deep learning was able to achieve high levels of accuracy, sensitivity, and/or specificity in detecting and/or classifying nodules when applied to pulmonary CT scans not from the LIDC-IDRI database.