Measuring the Effects of Education in Detecting Lung Cancer on Chest Radiographs: Utilization of a New Assessment Tool

Methodological evaluation of systematic reviews based on the use of artificial intelligence systems in chest radiography

INTRODUCTION

In recent years, systems that use artificial intelligence (AI) in medical imaging have been developed, such as the interpretation of chest X-ray to rule out pathology. This has produced an increase in systematic reviews (SR) published on this topic. This article aims to evaluate the methodological quality of SRs that use AI for the diagnosis of thoracic pathology by simple chest X-ray.

MATERIAL AND METHODS

SRs evaluating the use of AI systems for the automatic reading of chest X-ray were selected. Searches were conducted (from inception to May 2022): PubMed, EMBASE, and the Cochrane Database of Systematic Reviews. Two investigators selected the reviews. From each SR, general, methodological and transparency characteristics were extracted. The PRISMA statement for diagnostic tests (PRISMA-DTA) and AMSTAR-2 were used. A narrative synthesis of the evidence was performed. Protocol registry: Open Science Framework: https://osf.io/4b6u2/.

RESULTS

After applying the inclusion and exclusion criteria, 7 SRs were selected (mean of 36 included studies per review). All the included SRs evaluated "deep learning" systems in which chest X-ray was used for the diagnosis of infectious diseases. Only 2 (29%) SRs indicated the existence of a review protocol. None of the SRs specified the design of the included studies or provided a list of excluded studies with their justification. Six (86%) SRs mentioned the use of PRISMA or one of its extensions. The risk of bias assessment was performed in 4 (57%) SRs. One (14%) SR included studies with some validation of AI techniques. Five (71%) SRs presented results in favour of the diagnostic capacity of the intervention. All SRs were rated "critically low" following AMSTAR-2 criteria.

CONCLUSIONS

The methodological quality of SRs that use AI systems in chest radiography can be improved. The lack of compliance in some items of the tools used means that the SRs published in this field must be interpreted with caution.

Role of chest radiographs in early lung cancer detection

Background

Lung cancer is the second most common and the most fatal form of cancer. Although annual low-dose computed tomography is used as the primary method of cancer screening, it presents challenges regarding resources as well as potential health risks from radiation exposure. Chest radiography (CXR), though less effective, is used frequently and commonly. Moreover, often in clinical settings, CXR is the first imaging modality used; computed tomography is subsequently performed if abnormalities are detected on CXRs. This study examined whether controlling for distractors and time constraints, as well as side-by-side comparison of multiple CXRs in clinical settings can aid earlier detection of radiological abnormalities indicative of lung cancer lesions.

Methods

Thirty-two attending physicians in the Republic of Korea examined 1,750 radiographs of 50 lung cancer cases. Using “hot spot” technology, participants indicated the possible locations of cancer lesions on each radiograph. Subsequently, the same radiographs, cropped to focus the anatomical regions where lung cancers were diagnosed, were shown side-by-side to the participants. The participants were asked to identify the radiograph which first enabled the diagnosis of lung cancer and which first showed a possible lesion.

Results

Removal of systemic constraints alone significantly improved lesion identification by 221.72±9.69 days. Presenting radiographs side-by-side, cropped to relevant areas, had an additional significant and positive impact on cancer detection in both hidden and open areas on CXRs. Also, lesions were detected at smaller sizes and earlier than when actually diagnosed.

Conclusions

CXR with improved methods and settings provides an easily accessible and low-risk imaging method for earlier detection of lung cancer compared to current clinical imaging settings. Further, this study demonstrates the potential effectiveness of programs that allow side-by-side comparisons of cropped areas of multiple radiographs to detect radiological abnormalities.