Prototypes for content-based image retrieval in clinical practice

Impact of an online reference system on the diagnosis of rare or atypical abdominal tumors and lesions

The purpose of the present study is to evaluate whether an online reference system (ORS, STATdx Elsevier, Amsterdam, Netherlands) impacts finding the histologically confirmed diagnosis of rare or atypical abdominal tumors and lesions in radiologic imaging. In total, 101 patients with rare tumor entities or lesions and atypical manifestations of common tumors were enrolled retrospectively. Blinded readings were performed by four radiologists with varying levels of experience, who reported on: (a) correct diagnosis (CD), (b) time needed to find the diagnosis, and (c) diagnostic confidence, initially without followed by the assistance of the ORS. The experienced reader (3 years of experience post-residency, CD 49.5%), as well as the advanced reader with 1 year of experience post-residency (CD 43.6%), and a resident with 5 years of experience (CD 46.5%) made the correct diagnosis more frequently compared to the less experienced reader (CD 25.7%). A significant improvement in making the correct diagnosis was only achieved by the advanced reader, the resident with 5 years of experience (CD with ORS 58.4%; p < 0.001). The advanced reader with 1 year of experience post-residency improved slightly (CD ORS 47.5%). The experienced reader (CD ORS 50.5%) and the less experienced reader (CD ORS 27.7%) did not improve significantly. The overall subjective confidence increased significantly when ORS was used (3.2 ± 0.9 vs. 3.8 ± 0.9; p < 0.001). While the ORS had a positive impact on making the correct diagnosis throughout all readers, it favored radiologists with more clinical experience rather than inexperienced residents. Moreover, the ORS increased the diagnostic confidence of all radiologists significantly. In conclusion, the ORS had no significant impact on the diagnosis of rare or atypical abdominal tumors and lesions except for one reader. The greatest benefit is the increase in diagnostic confidence.

Impact of a content-based image retrieval system on the interpretation of chest CTs of patients with diffuse parenchymal lung disease

OBJECTIVES

Content-based image retrieval systems (CBIRS) are a new and potentially impactful tool for radiological reporting, but their clinical evaluation is largely missing. This study aimed at assessing the effect of CBIRS on the interpretation of chest CT scans from patients with suspected diffuse parenchymal lung disease (DPLD).

MATERIALS AND METHODS

A total of 108 retrospectively included chest CT scans with 22 unique, clinically and/or histopathologically verified diagnoses were read by eight radiologists (four residents, four attending, median years reading chest CT scans 2.1± 0.7 and 12 ± 1.8, respectively). The radiologists read and provided the suspected diagnosis at a certified radiological workstation to simulate clinical routine. Half of the readings were done without CBIRS and half with the additional support of the CBIRS. The CBIRS retrieved the most likely of 19 lung-specific patterns from a large database of 6542 thin-section CT scans and provided relevant information (e.g., a list of potential differential diagnoses).

RESULTS

Reading time decreased by 31.3% (p < 0.001) despite the radiologists searching for additional information more frequently when the CBIRS was available (154 [72%] vs. 95 [43%], p < 0.001). There was a trend towards higher overall diagnostic accuracy (42.2% vs 34.7%, p = 0.083) when the CBIRS was available.

CONCLUSION

The use of the CBIRS had a beneficial impact on the reading time of chest CT scans in cases with DPLD. In addition, both resident and attending radiologists were more likely to consult informational resources if they had access to the CBIRS. Further studies are needed to confirm the observed trend towards increased diagnostic accuracy with the use of a CBIRS in practice.

KEY POINTS

• A content-based image retrieval system for supporting the diagnostic process of reading chest CT scans can decrease reading time by 31.3% (p < 0.001). • The decrease in reading time was present despite frequent usage of the content-based image retrieval system. • Additionally, a trend towards higher diagnostic accuracy was observed when using the content-based image retrieval system (42.2% vs 34.7%, p = 0.083).

Brain MRI Pattern Recognition Translated to Clinical Scenarios

We explored the performance of structure-based computational analysis in four neurodegenerative conditions [Ataxia (AT, n = 16), Huntington's Disease (HD, n = 52), Alzheimer's Disease (AD, n = 66), and Primary Progressive Aphasia (PPA, n = 50)], all characterized by brain atrophy. The independent variables were the volumes of 283 anatomical areas, derived from automated segmentation of T1-high resolution brain MRIs. The segmentation based volumetric quantification reduces image dimensionality from the voxel level [on the order of O(10⁶)] to anatomical structures [O(10²)] for subsequent statistical analysis. We evaluated the effectiveness of this approach on extracting anatomical features, already described by human experience and a priori biological knowledge, in specific scenarios: (1) when pathologies were relatively homogeneous, with evident image alterations (e.g., AT); (2) when the time course was highly correlated with the anatomical changes (e.g., HD), an analogy for prediction; (3) when the pathology embraced heterogeneous phenotypes (e.g., AD) so the classification was less efficient but, in compensation, anatomical and clinical information were less redundant; and (4) when the entity was composed of multiple subgroups that had some degree of anatomical representation (e.g., PPA), showing the potential of this method for the clustering of more homogeneous phenotypes that can be of clinical importance. Using the structure-based quantification and simple linear classifiers (partial least square), we achieve 87.5 and 73% of accuracy on differentiating AT and pre-symptomatic HD patents from controls, respectively. More importantly, the anatomical features automatically revealed by the classifiers agreed with the patterns previously described on these pathologies. The accuracy was lower (68%) on differentiating AD from controls, as AD does not display a clear anatomical phenotype. On the other hand, the method identified PPA clinical phenotypes and their respective anatomical signatures. Although most of the data are presented here as proof of concept in simulated clinical scenarios, structure-based analysis was potentially effective in characterizing phenotypes, retrieving relevant anatomical features, predicting prognosis, and aiding diagnosis, with the advantage of being easily translatable to clinics and understandable biologically.

Large-scale retrieval for medical image analytics: A comprehensive review

Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.

A retrieval-based computer-aided diagnosis system for the characterization of liver lesions in CT scans

RATIONALE AND OBJECTIVES

To evaluate a computer-aided diagnosis (CADx) system for the characterization of liver lesions in computed tomography (CT) scans. The stand-alone predictive performance of the CADx system was assessed and compared to that of three radiologists who were provided with the same amount of image information to which the CADx system had access.

MATERIALS AND METHODS

The CADx system operates as an image search engine exploiting texture analysis of liver lesion image data for the lesion in question and lesions from a database. A region of interest drawn around an indeterminate liver lesion is used as input query. The CADx system retrieves lesions of similar histology (benign/malignant), density (hypodense/hyperdense), or type (cyst/hemangioma/metastasis). The system's performance was evaluated with leave-one-patient-out receiver operating characteristic area under the curve on 685 CT scans from 372 patients that contained 2325 liver lesions (193 <1 cm(3)). Sensitivity, specificity, and positive and negative predictive values were evaluated separately for subcentimeter lesions. Results were compared to those of three radiologists who rated 83 liver lesions (20 hemangiomas, 20 metastases, 20 cysts, 20 hepatocellular carcinomas, and 3 focal nodular hyperplasias) displaying only the liver.

RESULTS

The CADx system's leave-one-patient-out receiver operating characteristic area under the curve was 97.1% for density, 91.4% for histology, and 95.5% for lesion type. For subcentimeter lesions, input of additional semantic information improved the system's performance. The CADx system has been proved to significantly outperform radiologists in discriminating lesion histology and type, provided the radiologists have no access to information other than the image. The radiologists were most reliable in diagnosing hemangioma given the limited image data.

CONCLUSIONS

The CADx system under study discriminated reliably between various liver lesions, even outperforming radiologists when accessing the same image information and demonstrated promising performance in classifying subcentimeter lesions in particular.

Atlas-based neuroinformatics via MRI: harnessing information from past clinical cases and quantitative image analysis for patient care

With the ever-increasing amount of anatomical information radiologists have to evaluate for routine diagnoses, computational support that facilitates more efficient education and clinical decision making is highly desired. Despite the rapid progress of image analysis technologies for magnetic resonance imaging of the human brain, these methods have not been widely adopted for clinical diagnoses. To bring computational support into the clinical arena, we need to understand the decision-making process employed by well-trained clinicians and develop tools to simulate that process. In this review, we discuss the potential of atlas-based clinical neuroinformatics, which consists of annotated databases of anatomical measurements grouped according to their morphometric phenotypes and coupled with the clinical informatics upon which their diagnostic groupings are based. As these are indexed via parametric representations, we can use image retrieval tools to search for phenotypes along with their clinical metadata. The review covers the current technology, preliminary data, and future directions of this field.