Usefulness of computerized method for lung nodule detection in digital chest radiographs using temporal subtraction images

Evaluation of the clinical utility of temporal subtraction using bone suppression processing in digital chest radiography

Rationale and objectives

To evaluate the usefulness of temporal subtraction using the bone suppression method in digital chest radiography for the detection of pulmonary lesions.

Materials and methods

The images of 31 patients with pulmonary lesions and 19 normal cases were included in the study. Conventional and bone suppression temporal subtraction were performed in the 50 cases selected and used for an observer performance study. Five radiologists participated in the study, and the differences between using conventional and bone suppression temporal subtraction were assessed using jackknife free-response receiver operating characteristic analysis.

Results

The average figure-of-merit values for all radiologists increased significantly using the bone suppression method, from 0.619 (conventional) to 0.696 (p = 0.032). The average sensitivity for detecting pulmonary lesions improved from 67.9% to 75.4%, and the average number of false-positive per case decreased from 0.336 to 0.252 using bone suppression temporal subtraction.

Conclusion

Bone suppression temporal subtraction processing can assist with the detection of subtle pulmonary lesions in digital chest radiographs.

Detection of suspected brain infarctions on CT can be significantly improved with temporal subtraction images

OBJECTIVE

To assess whether temporal subtraction (TS) images of brain CT improve the detection of suspected brain infarctions.

METHODS

Study protocols were approved by our institutional review board, and informed consent was waived because of the retrospective nature of this study. Forty-two sets of brain CT images of 41 patients, each consisting of a pair of brain CT images scanned at two time points (previous and current) between January 2011 and November 2016, were collected for an observer performance study. The 42 sets consisted of 23 cases with a total of 77 newly developed brain infarcts or hyperdense artery signs confirmed by two radiologists who referred to additional clinical information and 19 negative control cases. To create TS images, the previous images were registered to the current images by partly using a non-rigid registration algorithm and then subtracted. Fourteen radiologists independently interpreted the images to identify the lesions with and without TS images with an interval of over 4 weeks. A figure of merit (FOM) was calculated along with the jackknife alternative free-response receiver-operating characteristic analysis. Sensitivity, number of false positives per case (FPC) and reading time were analyzed by the Wilcoxon signed-rank test.

RESULTS

The mean FOM increased from 0.528 to 0.737 with TS images (p < 0.0001). The mean sensitivity and FPC improved from 26.5% and 0.243 to 56.0% and 0.153 (p < 0.0001 and p = 0.239), respectively. The mean reading time was 173 s without TS and 170 s with TS (p = 0.925).

CONCLUSION

The detectability of suspected brain infarctions was significantly improved with TS CT images.

KEY POINTS

• Although it is established that MRI is superior to CT in the detection of strokes, the first choice of modality for suspected stroke patients is often CT. • An observer performance study with 14 radiologists was performed to evaluate whether temporal subtraction images derived from a non-rigid transformation algorithm can significantly improve the detectability of newly developed brain infarcts on CT. • Temporal subtraction images were shown to significantly improve the detectability of newly developed brain infarcts on CT.

Multi-reader multi-case studies using the area under the receiver operator characteristic curve as a measure of diagnostic accuracy: systematic review with a focus on quality of data reporting

INTRODUCTION

We examined the design, analysis and reporting in multi-reader multi-case (MRMC) research studies using the area under the receiver-operating curve (ROC AUC) as a measure of diagnostic performance.

METHODS

We performed a systematic literature review from 2005 to 2013 inclusive to identify a minimum 50 studies. Articles of diagnostic test accuracy in humans were identified via their citation of key methodological articles dealing with MRMC ROC AUC. Two researchers in consensus then extracted information from primary articles relating to study characteristics and design, methods for reporting study outcomes, model fitting, model assumptions, presentation of results, and interpretation of findings. Results were summarized and presented with a descriptive analysis.

RESULTS

Sixty-four full papers were retrieved from 475 identified citations and ultimately 49 articles describing 51 studies were reviewed and extracted. Radiological imaging was the index test in all. Most studies focused on lesion detection vs. characterization and used less than 10 readers. Only 6 (12%) studies trained readers in advance to use the confidence scale used to build the ROC curve. Overall, description of confidence scores, the ROC curve and its analysis was often incomplete. For example, 21 (41%) studies presented no ROC curve and only 3 (6%) described the distribution of confidence scores. Of 30 studies presenting curves, only 4 (13%) presented the data points underlying the curve, thereby allowing assessment of extrapolation. The mean change in AUC was 0.05 (-0.05 to 0.28). Non-significant change in AUC was attributed to underpowering rather than the diagnostic test failing to improve diagnostic accuracy.

CONCLUSIONS

Data reporting in MRMC studies using ROC AUC as an outcome measure is frequently incomplete, hampering understanding of methods and the reliability of results and study conclusions. Authors using this analysis should be encouraged to provide a full description of their methods and results.

Applications of ROC analysis in medical research: recent developments and future directions

With the growing focus on comparative effectiveness research and personalized medicine, receiver-operating characteristic analysis can continue to play an important role in health care decision making. Specific applications of receiver-operating characteristic analysis include predictive model assessment and validation, biomarker diagnostics, responder analysis in patient-reported outcomes, and comparison of alternative treatment options. The authors present a survey of the potential applications of the method and briefly review several relevant extensions. Given the level of attention paid to biomarker validation, personalized medicine and comparative effectiveness research, it is highly likely that the receiver-operating characteristic analysis will remain an important visual and analytic tool for medical research and evidence-based medicine in the foreseeable future.