CT Colonography: Role of a second reader CAD paradigm in the initial training of radiologists

Development and validation of an educational software based in artificial neural networks for training in radiology (JORCAD) through an interactive learning activity

The use of Computer Aided Detection (CAD) software has been previously documented as a valuable tool to improve specialist training in Radiology. This research assesses the utility of an educational software tool aimed to train residents in Radiology and other related medical specialties and students from Medicine degree. This in-house developed software, called JORCAD, integrates a CAD system based in Convolutional Neural Networks (CNNs) with annotated cases from radiological image databases. The methodology followed for software validation was expert judgement after completing an interactive learning activity. Participants received a theoretical session and a software usage tutorial and afterwards utilized the application in a dedicated workstation to analyze a series of proposed cases of thorax computed tomography (CT) and mammography. A total of 26 expert participants from the Radiology Department at Salamanca University Hospital (15 specialists and 11 residents) fulfilled the activity and evaluated different aspects through a series of surveys: software usability, case navigation tools, CAD module utility for learning and JORCAD educational capabilities. Participants also graded imaging cases to establish JORCAD usefulness for training radiology residents. According to the statistical analysis of survey results and expert cases scoring, along with their opinions, it can be concluded that JORCAD software is a useful tool for training future specialists. The combination of CAD with annotated cases from validated databases enhances learning, offering a second opinion and changing the usual training paradigm. Including software as JORCAD in residency training programs of Radiology and other medical specialties would have a positive effect on trainees' background knowledge.

Computer-based self-training for CT colonography with and without CAD

OBJECTIVES

To determine whether (1) computer-based self-training for CT colonography (CTC) improves interpretation performance of novice readers; (2) computer-aided detection (CAD) use during training affects learning.

METHODS

Institutional review board approval and patients' informed consent were obtained for all cases included in this study. Twenty readers (17 radiology residents, 3 radiologists) with no experience in CTC interpretation were recruited in three centres. After an introductory course, readers performed a baseline assessment test (37 cases) using CAD as second reader. Then they were randomized (1:1) to perform either a computer-based self-training (150 cases verified at colonoscopy) with CAD as second reader or the same training without CAD. The same assessment test was repeated after completion of the training programs. Main outcome was per lesion sensitivity (≥ 6 mm). A generalized estimating equation model was applied to evaluate readers' performance and the impact of CAD use during training.

RESULTS

After training, there was a significant improvement in average per lesion sensitivity in the unassisted phase, from 74% (356/480) to 83% (396/480) (p < 0.001), and in the CAD-assisted phase, from 83% (399/480) to 87% (417/480) (p = 0.021), but not in average per patient sensitivity, from 93% (390/420) to 94% (395/420) (p = 0.41), and specificity, from 81% (260/320) to 86% (276/320) (p = 0.15). No significant effect of CAD use during training was observed on per patient sensitivity and specificity, nor on per lesion sensitivity.

CONCLUSIONS

A computer-based self-training program for CTC improves readers' per lesion sensitivity. CAD as second reader does not have a significant impact on learning if used during training.

KEY POINTS

• Computer-based self-training for CT colonography improves per lesion sensitivity of novice readers. • Self-training program does not increase per patient specificity of novice readers. • CAD used during training does not have significant impact on learning.

Assessment of the Incremental Benefit of Computer-Aided Detection (CAD) for Interpretation of CT Colonography by Experienced and Inexperienced Readers

OBJECTIVES

To quantify the incremental benefit of computer-assisted-detection (CAD) for polyps, for inexperienced readers versus experienced readers of CT colonography.

METHODS

10 inexperienced and 16 experienced radiologists interpreted 102 colonography studies unassisted and with CAD utilised in a concurrent paradigm. They indicated any polyps detected on a study sheet. Readers' interpretations were compared against a ground-truth reference standard: 46 studies were normal and 56 had at least one polyp (132 polyps in total). The primary study outcome was the difference in CAD net benefit (a combination of change in sensitivity and change in specificity with CAD, weighted towards sensitivity) for detection of patients with polyps.

RESULTS

Inexperienced readers' per-patient sensitivity rose from 39.1% to 53.2% with CAD and specificity fell from 94.1% to 88.0%, both statistically significant. Experienced readers' sensitivity rose from 57.5% to 62.1% and specificity fell from 91.0% to 88.3%, both non-significant. Net benefit with CAD assistance was significant for inexperienced readers but not for experienced readers: 11.2% (95%CI 3.1% to 18.9%) versus 3.2% (95%CI -1.9% to 8.3%) respectively.

CONCLUSIONS

Concurrent CAD resulted in a significant net benefit when used by inexperienced readers to identify patients with polyps by CT colonography. The net benefit was nearly four times the magnitude of that observed for experienced readers. Experienced readers did not benefit significantly from concurrent CAD.

Development of a personalized training system using the Lung Image Database Consortium and Image Database resource Initiative Database

RATIONALE AND OBJECTIVES

The aim of this study was to develop a personalized training system using the Lung Image Database Consortium (LIDC) and Image Database resource Initiative (IDRI) Database, because collecting, annotating, and marking a large number of appropriate computed tomography (CT) scans, and providing the capability of dynamically selecting suitable training cases based on the performance levels of trainees and the characteristics of cases are critical for developing a efficient training system.

MATERIALS AND METHODS

A novel approach is proposed to develop a personalized radiology training system for the interpretation of lung nodules in CT scans using the Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) database, which provides a Content-Boosted Collaborative Filtering (CBCF) algorithm for predicting the difficulty level of each case of each trainee when selecting suitable cases to meet individual needs, and a diagnostic simulation tool to enable trainees to analyze and diagnose lung nodules with the help of an image processing tool and a nodule retrieval tool.

RESULTS

Preliminary evaluation of the system shows that developing a personalized training system for interpretation of lung nodules is needed and useful to enhance the professional skills of trainees.

CONCLUSIONS

The approach of developing personalized training systems using the LIDC/IDRL database is a feasible solution to the challenges of constructing specific training program in terms of cost and training efficiency.

Clinical evaluation of semi-automatic landmark-based lesion tracking software for CT-scans

Background

To evaluate a semi-automatic landmark-based lesion tracking software enabling navigation between RECIST lesions in baseline and follow-up CT-scans.

Methods

The software automatically detects 44 stable anatomical landmarks in each thoraco/abdominal/pelvic CT-scan, sets up a patient specific coordinate-system and cross-links the coordinate-systems of consecutive CT-scans. Accuracy of the software was evaluated on 96 RECIST lesions (target- and non-target lesions) in baseline and follow-up CT-scans of 32 oncologic patients (64 CT-scans). Patients had to present at least one thoracic, one abdominal and one pelvic RECIST lesion. Three radiologists determined the deviation between lesions’ centre and the software’s navigation result in consensus.

Results

The initial mean runtime of the system to synchronize baseline and follow-up examinations was 19.4 ± 1.2 seconds, with subsequent navigation to corresponding RECIST lesions facilitating in real-time. Mean vector length of the deviations between lesions’ centre and the semi-automatic navigation result was 10.2 ± 5.1 mm without a substantial systematic error in any direction. Mean deviation in the cranio-caudal dimension was 5.4 ± 4.0 mm, in the lateral dimension 5.2 ± 3.9 mm and in the ventro-dorsal dimension 5.3 ± 4.0 mm.

Conclusion

The investigated software accurately and reliably navigates between lesions in consecutive CT-scans in real-time, potentially accelerating and facilitating cancer staging.

Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography

Objective

To evaluate a computer-aided detection (CADe) system for lytic and blastic spinal metastases on computed tomography (CT).

Methods

We retrospectively evaluated the CADe system on 20 consecutive patients with 42 lytic and on 30 consecutive patients with 172 blastic metastases. The CADe system was trained using CT images of 114 subjects with 102 lytic and 308 blastic spinal metastases. Lesions were annotated by experienced radiologists. Detected benign lesions were considered false-positive findings. Detector sensitivity and the number of false-positive findings were calculated as the criteria for detector performance, and free-response receiver operating characteristic (FROC) analysis was conducted. Detailed analysis of false-positive and false-negative findings was performed.

Results

Algorithm runtime is 3 ± 0.5 min per patient. The system achieves a sensitivity of 83 % at 3.5 false positives per patient on average for blastic metastases and a sensitivity of 88 % at 3.7 false positives for lytic metastases. False positives appeared predominantly in the area of degenerative changes in the case of the blastic metastasis detector and in osteoporotic areas in the case of the lytic metastasis detector.

Conclusion

The CADe system reliably detects thoracolumbar spine metastases in real time. An additional study is planned to evaluate how the bone lesion CADe system improves radiologists’ accuracy and efficiency in a clinical setting.

Key Points

• Computer-aided detection (CADe) of bone metastases has been developed for spinal CT.

• The CADe system exhibits high sensitivity with a tolerable false-positive rate.

• Analysis of false-positive detection may further improve the system.

• CADe may reduce the number of missed spinal metastases at CT interpretation.

How to read and report CTC

Owing to encouraging results achieved in the clinical practice, CT colonography (CTC) is being increasingly employed for the examination of the whole colon and rectum and is quickly becoming a widely accepted diagnostic technique that is replacing double contrast barium enema and appears a promising tool for colorectal cancer screening as well. The increasing number of symptomatic and asymptomatic patients undergoing CTC for both evaluation of symptoms and colorectal cancer screening, along with the growing availability of CTC facilities in most healthcare departments and imaging centres, requires that a sufficient number of radiologists be adequately trained in performing and interpreting CTC studies. Indeed, optimal performance of CTC depends on a number of factors, including the quality of colonic preparation (e.g. laxative bowel cleansing and optimised colonic distension), the CTC image acquisition protocol used, and reading approach and specific skills of radiologists for correct detection and interpretation of colonic findings. Consequently, dedicated training and expertise is key to obtain high sensitivity in lesion detection and reduce the number of false positives, thus ensuring an optimal clinical management of patients. To this purpose, dedicated training programmes are essential to teach and standardise not only the approach to CTC reading, but also reporting of colonic findings.

Colonic polyps: inheritance, susceptibility, risk evaluation, and diagnostic management

Colorectal cancer (CRC) is the third-ranked neoplasm in order of incidence and mortality, worldwide, and the second cause of cancer death in industrialized countries. One of the most important environmental risk factors for CRC is a Western-type diet, which is characterized by a low-fiber and high-fat content. Up to 25% of patients with CRC have a family history for CRC, and a fraction of these patients are affected by hereditary syndromes, such as familial adenomatous polyposis, Gardner or Turcot syndromes, or hereditary nonpolyposis colorectal cancer. The onset of CRC is triggered by a well-defined combination of genetic alterations, which form the bases of the adenoma-carcinoma sequence hypothesis and justify the set-up of CRC screening techniques. Several screening and diagnostic tests for CRC are illustrated, including rectosigmoidoscopy, optical colonoscopy (OC), double contrast barium enema (DCBE), and computed tomography colonography (CTC). The strengths and weaknesses of each technique are discussed. Particular attention is paid to CTC, which has evolved from an experimental technique to an accurate and mature diagnostic approach, and gained wide acceptance and clinical validation for CRC screening. This success of CTC is due mainly to its ability to provide cross-sectional analytical images of the entire colon and secondarily detect extracolonic findings, with minimal invasiveness and lower cost than OC, and with greater detail and diagnostic accuracy than DCBE. Moreover, especially with the advent and widespread availability of modern multidetector CT scanners, excellent quality 2D and 3D reconstructions of the large bowel can be obtained routinely with a relatively low radiation dose. Computer-aided detection systems have also been developed to assist radiologists in reading CTC examinations, improving overall diagnostic accuracy and potentially speeding up the clinical workflow of CTC image interpretation.