Automated temporal tracking and segmentation of lymphoma on serial CT examinations

An automated methodology for whole-body, multimodality tracking of individual cancer lesions

Objective. Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities.Approach. This study introduces a lesion tracking methodology and benchmarked it using 2368Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance.Main results. Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes).Significance. The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.

Translation of AI into oncology clinical practice

Artificial intelligence (AI) is a transformative technology that is capturing popular imagination and can revolutionize biomedicine. AI and machine learning (ML) algorithms have the potential to break through existing barriers in oncology research and practice such as automating workflow processes, personalizing care, and reducing healthcare disparities. Emerging applications of AI/ML in the literature include screening and early detection of cancer, disease diagnosis, response prediction, prognosis, and accelerated drug discovery. Despite this excitement, only few AI/ML models have been properly validated and fewer have become regulated products for routine clinical use. In this review, we highlight the main challenges impeding AI/ML clinical translation. We present different clinical use cases from the domains of radiology, radiation oncology, immunotherapy, and drug discovery in oncology. We dissect the unique challenges and opportunities associated with each of these cases. Finally, we summarize the general requirements for successful AI/ML implementation in the clinic, highlighting specific examples and points of emphasis including the importance of multidisciplinary collaboration of stakeholders, role of domain experts in AI augmentation, transparency of AI/ML models, and the establishment of a comprehensive quality assurance program to mitigate risks of training bias and data drifts, all culminating toward safer and more beneficial AI/ML applications in oncology labs and clinics.

Development and validation of a longitudinal soft-tissue metastatic lesion matching algorithm

Metastatic cancer presents with many, sometimes hundreds of metastatic lesions through the body, which often respond heterogeneously to treatment. Therefore, lesion-level assessment is necessary for a complete understanding of disease response. Lesion-level assessment typically requires manual matching of corresponding lesions, which is a tedious, subjective, and error-prone task. This study introduces a fully automated algorithm for matching of metastatic lesions in longitudinal medical images. The algorithm entails four steps: (1) image registration, (2) lesion dilation, (3) lesion clustering, and (4) linear assignment. In step (1), 3D deformable registration is used to register the scans. In step (2), lesion contours are conformally dilated. In step (3), lesion clustering is evaluated based on local metrics. In step (4), matching is assigned based on non-greedy cost minimization. The algorithm was optimized (e.g. choice of deformable registration algorithm, dilatation size) and validated on 140 scan-pairs of 32 metastatic cancer patients from two independent clinical trials, who received longitudinal PET/CT scans as part of their treatment response assessment. Registration error was evaluated using landmark distance. A sensitivity study was performed to evaluate the optimal lesion dilation magnitude. Lesion matching performance accuracy was evaluated for all patients and for a subset with high disease burden. Two investigated deformable registration approaches (whole body deformable and articulated deformable registrations) led to similar performance with the overall registration accuracy between 2.3 and 2.6 mm. The optimal dilation magnitude of 25 mm yielded almost a perfect matching accuracy of 0.98. No significant matching accuracy decrease was observed in the subset of patients with high lesion disease burden. In summary, lesion matching using our new algorithm was highly accurate and a significant improvement, when compared to previously established methods. The proposed method enables accurate automated metastatic lesion matching in whole-body longitudinal scans.

Mediastinal lymph node detection and station mapping on chest CT using spatial priors and random forest

PURPOSE

To develop an automated system for mediastinal lymph node detection and station mapping for chest CT.

METHODS

The contextual organs, trachea, lungs, and spine are first automatically identified to locate the region of interest (ROI) (mediastinum). The authors employ shape features derived from Hessian analysis, local object scale, and circular transformation that are computed per voxel in the ROI. Eight more anatomical structures are simultaneously segmented by multiatlas label fusion. Spatial priors are defined as the relative multidimensional distance vectors corresponding to each structure. Intensity, shape, and spatial prior features are integrated and parsed by a random forest classifier for lymph node detection. The detected candidates are then segmented by the following curve evolution process. Texture features are computed on the segmented lymph nodes and a support vector machine committee is used for final classification. For lymph node station labeling, based on the segmentation results of the above anatomical structures, the textual definitions of mediastinal lymph node map according to the International Association for the Study of Lung Cancer are converted into patient-specific color-coded CT image, where the lymph node station can be automatically assigned for each detected node.

RESULTS

The chest CT volumes from 70 patients with 316 enlarged mediastinal lymph nodes are used for validation. For lymph node detection, their system achieves 88% sensitivity at eight false positives per patient. For lymph node station labeling, 84.5% of lymph nodes are correctly assigned to their stations.

CONCLUSIONS

Multiple-channel shape, intensity, and spatial prior features aggregated by a random forest classifier improve mediastinal lymph node detection on chest CT. Using the location information of segmented anatomic structures from the multiatlas formulation enables accurate identification of lymph node stations.

Imaging genomics in cancer research: limitations and promises

Recently, radiogenomics or imaging genomics has emerged as a novel high-throughput method of associating imaging features with genomic data. Radiogenomics has the potential to provide comprehensive intratumour, intertumour and peritumour information non-invasively. This review article summarizes the current state of radiogenomic research in tumour characterization, discusses some of its limitations and promises and projects its future directions. Semi-radiogenomic studies that relate specific gene expressions to imaging features will also be briefly reviewed.

Automated method for detection and segmentation of liver metastatic lesions in follow-up CT examinations

This paper presents a fully automated method for detection and segmentation of liver metastases in serial computed tomography (CT) examinations. Our method uses a given two-dimensional baseline segmentation mask for identifying the lesion location in the follow-up CT and locating surrounding tissues, using nonrigid image registration and template matching, in order to reduce the search area for segmentation. Adaptive region growing and mean-shift clustering are used to obtain the lesion segmentation. Our database contains 127 cases from the CT abdomen unit at Sheba Medical Center. Development of the methodology was conducted using 22 of the cases, and testing was conducted on the remaining 105 cases. Results show that 94 of the 105 lesions were detected, for an overall matching rate of 90% making the correct RECIST 1.1 assessment in 88% of the cases. The average Dice index was [Formula: see text], the average sensitivity was [Formula: see text], and the positive predictive value was [Formula: see text]. In 92% of the rated cases, the results were classified by the radiologists as acceptable or better. The segmentation performance, matching rate, and RECIST assessment results hence appear promising.

Classification of thresholded regions based on selective use of PET, CT and PET-CT image features

Fluorodeoxyglucose positron emission tomography - computed tomography (FDG PET-CT) is the preferred image modality for lymphoma diagnosis. Sites of disease generally appear as foci of increased FDG uptake. Thresholding methods are often applied to robustly separate these regions. However, its main limitation is that it also includes sites of FDG excretion and physiological FDG uptake regions, which we define as FEPU - sites of FEPU include the bladder, renal, papillae, ureters, brain, heart and brown fat. FEPU can make image interpretation problematic. The ability to identify and label FEPU sites and separate them from abnormal regions is an important process that could improve image interpretation. We propose a new method to automatically separate and label FEPU sites from the thresholded PET images. Our method is based on the selective use of features extracted from data types comprising of PET, CT and PET-CT. Our FEPU classification of 43 clinical lymphoma patient studies revealed higher accuracy when compared to non-selective image features.

Semiautomatic segmentation and follow-up of multicomponent low-grade tumors in longitudinal brain MRI studies

PURPOSE

Tracking the progression of low grade tumors (LGTs) is a challenging task, due to their slow growth rate and associated complex internal tumor components, such as heterogeneous enhancement, hemorrhage, and cysts. In this paper, the authors show a semiautomatic method to reliably track the volume of LGTs and the evolution of their internal components in longitudinal MRI scans.

METHODS

The authors' method utilizes a spatiotemporal evolution modeling of the tumor and its internal components. Tumor components gray level parameters are estimated from the follow-up scan itself, obviating temporal normalization of gray levels. The tumor delineation procedure effectively incorporates internal classification of the baseline scan in the time-series as prior data to segment and classify a series of follow-up scans. The authors applied their method to 40 MRI scans of ten patients, acquired at two different institutions. Two types of LGTs were included: Optic pathway gliomas and thalamic astrocytomas. For each scan, a "gold standard" was obtained manually by experienced radiologists. The method is evaluated versus the gold standard with three measures: gross total volume error, total surface distance, and reliability of tracking tumor components evolution.

RESULTS

Compared to the gold standard the authors' method exhibits a mean Dice similarity volumetric measure of 86.58% and a mean surface distance error of 0.25 mm. In terms of its reliability in tracking the evolution of the internal components, the method exhibits strong positive correlation with the gold standard.

CONCLUSIONS

The authors' method provides accurate and repeatable delineation of the tumor and its internal components, which is essential for therapy assessment of LGTs. Reliable tracking of internal tumor components over time is novel and potentially will be useful to streamline and improve follow-up of brain tumors, with indolent growth and behavior.

Automated tracking of quantitative assessments of tumor burden in clinical trials

THERE ARE TWO KEY CHALLENGES HINDERING EFFECTIVE USE OF QUANTITATIVE ASSESSMENT OF IMAGING IN CANCER RESPONSE ASSESSMENT: 1) Radiologists usually describe the cancer lesions in imaging studies subjectively and sometimes ambiguously, and 2) it is difficult to repurpose imaging data, because lesion measurements are not recorded in a format that permits machine interpretation and interoperability. We have developed a freely available software platform on the basis of open standards, the electronic Physician Annotation Device (ePAD), to tackle these challenges in two ways. First, ePAD facilitates the radiologist in carrying out cancer lesion measurements as part of routine clinical trial image interpretation workflow. Second, ePAD records all image measurements and annotations in a data format that permits repurposing image data for analyses of alternative imaging biomarkers of treatment response. To determine the impact of ePAD on radiologist efficiency in quantitative assessment of imaging studies, a radiologist evaluated computed tomography (CT) imaging studies from 20 subjects having one baseline and three consecutive follow-up imaging studies with and without ePAD. The radiologist made measurements of target lesions in each imaging study using Response Evaluation Criteria in Solid Tumors 1.1 criteria, initially with the aid of ePAD, and then after a 30-day washout period, the exams were reread without ePAD. The mean total time required to review the images and summarize measurements of target lesions was 15% (P < .039) shorter using ePAD than without using this tool. In addition, it was possible to rapidly reanalyze the images to explore lesion cross-sectional area as an alternative imaging biomarker to linear measure. We conclude that ePAD appears promising to potentially improve reader efficiency for quantitative assessment of CT examinations, and it may enable discovery of future novel image-based biomarkers of cancer treatment response.

Consistency and efficiency of CT analysis of metastatic disease: semiautomated lesion management application within a PACS

OBJECTIVE

The purpose of this study was to evaluate the success, consistency, and efficiency of a semiautomated lesion management application within a PACS in the analysis of metastatic lesions in serial CT examinations of cancer patients.

MATERIALS AND METHODS

Two observers using baseline and follow-up CT data independently reviewed 93 target lesions (17 lung, five liver, 71 lymph node) in 50 patients with either metastatic bladder or prostate cancer. The observers measured the longest axis (or short axis for lymph nodes) of each lesion and made Response Evaluation Criteria in Solid Tumors (RECIST) determinations using manual and lesion management application methods. The times required for examination review, RECIST calculations, and data input were recorded. The Wilcoxon signed rank test was used to assess time differences, and Bland-Altman analysis was used to assess interobserver agreement within the manual and lesion management application methods. Percentage success rates were also reported.

RESULTS

With the lesion management application, most lung and liver lesions were semiautomatically segmented. Comparison of the lesion management application and manual methods for all lesions showed a median time saving of 45% for observer 1 (p<0.05) and 28% for observer 2 (p=0.05) on follow-up scans versus 28% for observer 1 (p<0.05) and 9% for observer 2 (p=0.087) on baseline scans. Variability of measurements showed mean percentage change differences of only 8.9% for the lesion management application versus 26.4% for manual measurements.

CONCLUSION

With the lesion management application method, most lung and liver lesions were successfully segmented semiautomatically; the results were more consistent between observers; and assessment of tumor size was faster than with the manual method.

Snake model-based lymphoma segmentation for sequential CT images

The measurement of the size of lesions in follow-up CT examinations of cancer patients is important to evaluate the success of treatment. This paper presents an automatic algorithm for identifying and segmenting lymph nodes in CT images across longitudinal time points. Firstly, a two-step image registration method is proposed to locate the lymph nodes including coarse registration based on body region detection and fine registration based on a double-template matching algorithm. Then, to make the initial segmentation approximate the boundaries of lymph nodes, the initial image registration result is refined with intensity and edge information. Finally, a snake model is used to evolve the refined initial curve and obtain segmentation results. Our algorithm was tested on 26 lymph nodes at multiple time points from 14 patients. The image at the earlier time point was used as the baseline image to be used in evaluating the follow-up image, resulting in 76 total test cases. Of the 76 test cases, we made a 76 (100%) successful detection and 38/40 (95%) correct clinical assessment according to Response Evaluation Criteria in Solid Tumors (RECIST). The quantitative evaluation based on several metrics, such as average Hausdorff distance, indicates that our algorithm is produces good results. In addition, the proposed algorithm is fast with an average computing time 2.58s. The proposed segmentation algorithm for lymph nodes is fast and can achieve high segmentation accuracy, which may be useful to automate the tracking and evaluation of cancer therapy.

Automated registration, segmentation, and measurement of metastatic melanoma tumors in serial CT scans

OBJECTIVES

Our goal was to evaluate a new software capability that integrates registration, segmentation and tumor measurement across serial exams within a picture archiving communication system (PACS) to expedite tumor measurement.

MATERIALS AND METHODS

Patients treated under institutional review board-approved protocols for metastatic melanoma were retrospectively reviewed. Of the 19 included patients, five were male, the median age was 43.2, and all received treatment using an adoptive cell therapy. Seventy-one lung, liver, and subcutaneous tumors were manually measured using RECIST (Response Evaluation Criteria In Solid Tumors) criteria before therapy (baseline computed tomography [CT]) and within 3 months after therapy (follow-up CT). We performed semiautomated registration, segmentation, and RECIST measurements at both time points within PACS (Carestream Health, Rochester, NY). We compared manual and software-generated RECIST measurements using Bland-Altman plots.

RESULTS

The median manually measured RECIST diameter for all baseline tumors was 2.1 (1.0-6.2) cm. The refined registration function identified 70/71 (98.6%) tumors on the follow-up CT. On the baseline CT, all 21 liver, 27/32 (84%) lung, and 10/18 (55%) subcutaneous tumors completed segmentation. On the follow-up CT, 19/21 (90%) liver, 21/27 (78%) lung, and 8/10 (80%) subcutaneous tumors completed segmentation. The Bland-Altman plot demonstrated a 95% confidence interval of ±0.7 cm when comparing the software-generated and manual RECIST measurements.

CONCLUSIONS

The PACS software performed semiautomated baseline tumor measurements and fully automated follow-up tumor measurements in a majority of lung, liver, and subcutaneous tumors. In our patients, semiautomated metastatic tumor measurement did not obviate the need for physician oversight due to disease and treatment-related factors.

Tracking Metastatic Brain Tumors in Longitudinal Scans via Joint Image Registration and Labeling

The treatment of metastatic brain tumors with stereotactic radiosurgery requires that the clinician first locate the tumors and measure their volumes. Thoroughly searching a patient scan for brain tumors and delineating the lesions can be a long and difficult task when done manually and is also prone to human error. In this paper, we present an automated method for detecting changes in brain tumor lesions over longitudinal scans to aide the clinician's task of determining tumor volumes. Our approach jointly registers the current image with a previous scan while estimating changes in intensity correspondences due to tumor growth or regression. We combine the label map with correspondence changes with tumor segmentations from a previous scan to estimate the metastases in the new image. Alignment and tumor tracking results show promise on 28 registrations using real patient data.