1
|
Veroli BD, Lederman R, Shoshan Y, Sosna J, Joskowicz L. A graph-theoretic approach for the analysis of lesion changes and lesions detection review in longitudinal oncological imaging. Med Image Anal 2024; 97:103268. [PMID: 39029156 DOI: 10.1016/j.media.2024.103268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 05/27/2024] [Accepted: 07/08/2024] [Indexed: 07/21/2024]
Abstract
Radiological follow-up of oncology patients requires the detection of lesions and the quantitative analysis of lesion changes in longitudinal imaging studies of patients, which is time-consuming and requires expertise. We present here a new method and workflow for the analysis and review of lesions and volumetric lesion changes in longitudinal scans of a patient. The generic graph-based method consists of lesion matching, classification of changes in individual lesions, and detection of patterns of lesion changes computed from the properties of the graph and its connected components. The workflow guides clinicians in the detection of missed lesions and wrongly identified lesions in manual and computed lesion annotations using the analysis of lesion changes. It serves as a heuristic method for the automatic revision of ground truth lesion annotations in longitudinal scans. The methods were evaluated on longitudinal studies of patients with three or more examinations of metastatic lesions in the lung (19 patients, 83 CT scans, 1178 lesions), the liver (18 patients, 77 CECT scans, 800 lesions) and the brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions) with ground-truth lesion annotations. Lesion matching yielded a precision of 0.92-1.0 and recall of 0.91-0.99. The classification of changes in individual lesions yielded an accuracy of 0.87-0.97. The classification of patterns of lesion changes yielded an accuracy of 0.80-0.94. The lesion detection review workflow applied to manual and computed lesion annotations yielded 120 and 55 missed lesions and 20 and 164 wrongly identified lesions for all longitudinal studies of patients, respectively. The automatic analysis of lesion changes and review of lesion detection in longitudinal studies of oncological patients helps detect missed lesions and wrongly identified lesions. This method may help improve the accuracy of radiological interpretation and the disease status evaluation.
Collapse
Affiliation(s)
- Beniamin Di Veroli
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel
| | - Richard Lederman
- Department of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Yigal Shoshan
- Department of Neurosurgery, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Jacob Sosna
- Department of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel; Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Israel.
| |
Collapse
|
2
|
Joskowicz L, Di Veroli B, Lederman R, Shoshan Y, Sosna J. Three scans are better than two for follow-up: An automatic method for finding missed and misidentified lesions in cross-sectional follow-up of oncology patients. Eur J Radiol 2024; 176:111530. [PMID: 38810439 DOI: 10.1016/j.ejrad.2024.111530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2024] [Revised: 05/13/2024] [Accepted: 05/24/2024] [Indexed: 05/31/2024]
Abstract
PURPOSE Missed and misidentified neoplastic lesions in longitudinal studies of oncology patients are pervasive and may affect the evaluation of the disease status. Two newly identified patterns of lesion changes, lone lesions and non-consecutive lesion changes, may help radiologists to detect these lesions. This study evaluated a new interpretation revision workflow of lesion annotations in three or more consecutive scans based on these suspicious patterns. METHODS The interpretation revision workflow was evaluated on manual and computed lesion annotations in longitudinal oncology patient studies. For the manual revision, a senior radiologist and a senior neurosurgeon (the readers) manually annotated the lesions in each scan and later revised their annotations to identify missed and misidentified lesions with the workflow using the automatically detected patterns. For the computerized revision, lesion annotations were first computed with a previously trained nnU-Net and were then automatically revised with an AI-based method that automates the workflow readers' decisions. The evaluation included 67 patient studies with 2295 metastatic lesions in lung (19 patients, 83 CT scans, 1178 lesions), liver (18 patients, 77 CECT scans, 800 lesions) and brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions). RESULTS Revision of the manual lesion annotations revealed 120 missed lesions and 20 misidentified lesions in 31 out of 67 (46%) studies. The automatic revision reduced the number of computed missed lesions by 55 and computed misidentified lesions by 164 in 51 out of 67 (76%) studies. CONCLUSION Automatic analysis of three or more consecutive volumetric scans helps find missed and misidentified lesions and may improve the evaluation of temporal changes of oncological lesions.
Collapse
Affiliation(s)
- Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel.
| | - Beniamin Di Veroli
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel
| | - Richard Lederman
- Dept of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Yigal Shoshan
- Dept of Neurosurgery, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Jacob Sosna
- Dept of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| |
Collapse
|
3
|
Santoro-Fernandes V, Huff DT, Rivetti L, Deatsch A, Schott B, Perlman SB, Jeraj R. An automated methodology for whole-body, multimodality tracking of individual cancer lesions. Phys Med Biol 2024; 69:085012. [PMID: 38457838 DOI: 10.1088/1361-6560/ad31c6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 03/08/2024] [Indexed: 03/10/2024]
Abstract
Objective. Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities.Approach. This study introduces a lesion tracking methodology and benchmarked it using 2368Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance.Main results. Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes).Significance. The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.
Collapse
Affiliation(s)
- Victor Santoro-Fernandes
- School of Medicine and Public Health, Department of Medical Physics, University of Wisconsin, Madison, WI, United States of America
| | - Daniel T Huff
- School of Medicine and Public Health, Department of Medical Physics, University of Wisconsin, Madison, WI, United States of America
| | - Luciano Rivetti
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Alison Deatsch
- School of Medicine and Public Health, Department of Medical Physics, University of Wisconsin, Madison, WI, United States of America
| | - Brayden Schott
- School of Medicine and Public Health, Department of Medical Physics, University of Wisconsin, Madison, WI, United States of America
| | - Scott B Perlman
- School of Medicine and Public Health, Department of Radiology, Section of Nuclear Medicine, University of Wisconsin, Madison, WI, United States of America
| | - Robert Jeraj
- School of Medicine and Public Health, Department of Medical Physics, University of Wisconsin, Madison, WI, United States of America
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| |
Collapse
|
4
|
Rochman S, Szeskin A, Lederman R, Sosna J, Joskowicz L. Graph-based automatic detection and classification of lesion changes in pairs of CT studies for oncology follow-up. Int J Comput Assist Radiol Surg 2024; 19:241-251. [PMID: 37540449 DOI: 10.1007/s11548-023-03000-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Accepted: 07/13/2023] [Indexed: 08/05/2023]
Abstract
PURPOSE Radiological follow-up of oncology patients requires the quantitative analysis of lesion changes in longitudinal imaging studies, which is time-consuming, requires expertise, and is subject to variability. This paper presents a comprehensive graph-based method for the automatic detection and classification of lesion changes in current and prior CT scans. METHODS The inputs are the current and prior CT scans and their organ and lesion segmentations. Classification of lesion changes is formalized as bipartite graph matching where lesion pairings are computed by adaptive overlap-based lesion matching. Six types of lesion changes are computed by connected components analysis. The method was evaluated on 208 pairs of lung and liver CT scans from 57 patients with 4600 lesions, 1713 lesion matchings and 2887 lesion changes. Ground-truth lesion segmentations, lesion matchings and lesion changes were created by an expert radiologist. RESULTS Our method yields a lesion matching rate accuracy of 99.7% (394/395) and 95.0% (1252/1318) for the lung and liver datasets. Precision and recall are > 0.99 and 0.94 and 0.95 (respectively) for the detection of lesion changes. The analysis of lesion changes helped the radiologist detect 48 missed lesions and 8 spurious lesions in the input ground-truth lesion datasets. CONCLUSION The classification of lesion classification provides the clinician with a readily accessible and intuitive identification and classification of the lesion changes and their patterns in support of clinical decision making. Comprehensive automatic computer-aided lesion matching and analysis of lesion changes may improve quantitative follow-up and evaluation of disease status, assessment of treatment efficacy and response to therapy.
Collapse
Affiliation(s)
- Shalom Rochman
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Edmond J. Safra Campus, Givat Ram, 9190401, Jerusalem, Israel
| | - Adi Szeskin
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Edmond J. Safra Campus, Givat Ram, 9190401, Jerusalem, Israel
| | - Richard Lederman
- Department of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Jacob Sosna
- Department of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Edmond J. Safra Campus, Givat Ram, 9190401, Jerusalem, Israel.
| |
Collapse
|
5
|
Hammer Y, Najjar W, Kahanov L, Joskowicz L, Shoshan Y. Two is better than one: longitudinal detection and volumetric evaluation of brain metastases after Stereotactic Radiosurgery with a deep learning pipeline. J Neurooncol 2024; 166:547-555. [PMID: 38300389 PMCID: PMC10876809 DOI: 10.1007/s11060-024-04580-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 01/18/2024] [Indexed: 02/02/2024]
Abstract
PURPOSE Close MRI surveillance of patients with brain metastases following Stereotactic Radiosurgery (SRS) treatment is essential for assessing treatment response and the current disease status in the brain. This follow-up necessitates the comparison of target lesion sizes in pre- (prior) and post-SRS treatment (current) T1W-Gad MRI scans. Our aim was to evaluate SimU-Net, a novel deep-learning model for the detection and volumetric analysis of brain metastases and their temporal changes in paired prior and current scans. METHODS SimU-Net is a simultaneous multi-channel 3D U-Net model trained on pairs of registered prior and current scans of a patient. We evaluated its performance on 271 pairs of T1W-Gad MRI scans from 226 patients who underwent SRS. An expert oncological neurosurgeon manually delineated 1,889 brain metastases in all the MRI scans (1,368 with diameters > 5 mm, 834 > 10 mm). The SimU-Net model was trained/validated on 205 pairs from 169 patients (1,360 metastases) and tested on 66 pairs from 57 patients (529 metastases). The results were then compared to the ground truth delineations. RESULTS SimU-Net yielded a mean (std) detection precision and recall of 1.00±0.00 and 0.99±0.06 for metastases > 10 mm, 0.90±0.22 and 0.97±0.12 for metastases > 5 mm of, and 0.76±0.27 and 0.94±0.16 for metastases of all sizes. It improves lesion detection precision by 8% for all metastases sizes and by 12.5% for metastases < 10 mm with respect to standalone 3D U-Net. The segmentation Dice scores were 0.90±0.10, 0.89±0.10 and 0.89±0.10 for the above metastases sizes, all above the observer variability of 0.80±0.13. CONCLUSION Automated detection and volumetric quantification of brain metastases following SRS have the potential to enhance the assessment of treatment response and alleviate the clinician workload.
Collapse
Affiliation(s)
- Yonny Hammer
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Edmond. J. Safra Campus, Givat Ram, 9190401, Jerusalem, Israel
| | - Wenad Najjar
- Department of Neurosurgery, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Lea Kahanov
- Department of Neurosurgery, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| | - Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Edmond. J. Safra Campus, Givat Ram, 9190401, Jerusalem, Israel.
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.
| | - Yigal Shoshan
- Department of Neurosurgery, Hadassah Hebrew University Medical Center, Jerusalem, Israel
| |
Collapse
|
6
|
Joskowicz L, Szeskin A, Rochman S, Dodi A, Lederman R, Fruchtman-Brot H, Azraq Y, Sosna J. Follow-up of liver metastases: a comparison of deep learning and RECIST 1.1. Eur Radiol 2023; 33:9320-9327. [PMID: 37480549 DOI: 10.1007/s00330-023-09926-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2023] [Revised: 04/25/2023] [Accepted: 05/14/2023] [Indexed: 07/24/2023]
Abstract
OBJECTIVES To compare liver metastases changes in CT assessed by radiologists using RECIST 1.1 and with aided simultaneous deep learning-based volumetric lesion changes analysis. METHODS A total of 86 abdominal CT studies from 43 patients (prior and current scans) of abdominal CT scans of patients with 1041 liver metastases (mean = 12.1, std = 11.9, range 1-49) were analyzed. Two radiologists performed readings of all pairs; conventional with RECIST 1.1 and with computer-aided assessment. For computer-aided reading, we used a novel simultaneous multi-channel 3D R2U-Net classifier trained and validated on other scans. The reference was established by having an expert radiologist validate the computed lesion detection and segmentation. The results were then verified and modified as needed by another independent radiologist. The primary outcome measure was the disease status assessment with the conventional and the computer-aided readings with respect to the reference. RESULTS For conventional and computer-aided reading, there was a difference in disease status classification in 40 out of 86 (46.51%) and 10 out of 86 (27.9%) CT studies with respect to the reference, respectively. Computer-aided reading improved conventional reading in 30 CT studies by 34.5% for two readers (23.2% and 46.51%) with respect to the reference standard. The main reason for the difference between the two readings was lesion volume differences (p = 0.01). CONCLUSIONS AI-based computer-aided analysis of liver metastases may improve the accuracy of the evaluation of neoplastic liver disease status. CLINICAL RELEVANCE STATEMENT AI may aid radiologists to improve the accuracy of evaluating changes over time in metastasis of the liver. KEY POINTS • Classification of liver metastasis changes improved significantly in one-third of the cases with an automatically generated comprehensive lesion and lesion changes report. • Simultaneous deep learning changes detection and volumetric assessment may improve the evaluation of liver metastases temporal changes potentially improving disease management.
Collapse
Affiliation(s)
- Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Adi Szeskin
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Shalom Rochman
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Aviv Dodi
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Richard Lederman
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel
| | - Hila Fruchtman-Brot
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel
| | - Yusef Azraq
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel
| | - Jacob Sosna
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel.
| |
Collapse
|
7
|
Huff DT, Santoro-Fernandes V, Chen S, Chen M, Kashuk C, Weisman AJ, Jeraj R, Perk TG. Performance of an automated registration-based method for longitudinal lesion matching and comparison to inter-reader variability. Phys Med Biol 2023; 68:175031. [PMID: 37567220 PMCID: PMC10461173 DOI: 10.1088/1361-6560/acef8f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 07/25/2023] [Accepted: 08/11/2023] [Indexed: 08/13/2023]
Abstract
Objective.Patients with metastatic disease are followed throughout treatment with medical imaging, and accurately assessing changes of individual lesions is critical to properly inform clinical decisions. The goal of this work was to assess the performance of an automated lesion-matching algorithm in comparison to inter-reader variability (IRV) of matching lesions between scans of metastatic cancer patients.Approach.Forty pairs of longitudinal PET/CT and CT scans were collected and organized into four cohorts: lung cancers, head and neck cancers, lymphomas, and advanced cancers. Cases were also divided by cancer burden: low-burden (<10 lesions), intermediate-burden (10-29), and high-burden (30+). Two nuclear medicine physicians conducted independent reviews of each scan-pair and manually matched lesions. Matching differences between readers were assessed to quantify the IRV of lesion matching. The two readers met to form a consensus, which was considered a gold standard and compared against the output of an automated lesion-matching algorithm. IRV and performance of the automated method were quantified using precision, recall, F1-score, and the number of differences.Main results.The performance of the automated method did not differ significantly from IRV for any metric in any cohort (p> 0.05, Wilcoxon paired test). In high-burden cases, the F1-score (median [range]) was 0.89 [0.63, 1.00] between the automated method and reader consensus and 0.93 [0.72, 1.00] between readers. In low-burden cases, F1-scores were 1.00 [0.40, 1.00] and 1.00 [0.40, 1.00], for the automated method and IRV, respectively. Automated matching was significantly more efficient than either reader (p< 0.001). In high-burden cases, median matching time for the readers was 60 and 30 min, respectively, while automated matching took a median of 3.9 minSignificance.The automated lesion-matching algorithm was successful in performing lesion matching, meeting the benchmark of IRV. Automated lesion matching can significantly expedite and improve the consistency of longitudinal lesion-matching.
Collapse
Affiliation(s)
- Daniel T Huff
- AIQ Solutions, Madison, WI, United States of America
| | - Victor Santoro-Fernandes
- University of Wisconsin-Madison, Department of Medical Physics, Madison, WI, United States of America
| | - Song Chen
- The First Hospital of China Medical University, Department of Nuclear Medicine, Shenyang, Liaoning, CN, People’s Republic of China
| | - Meijie Chen
- The First Hospital of China Medical University, Department of Nuclear Medicine, Shenyang, Liaoning, CN, People’s Republic of China
| | - Carl Kashuk
- AIQ Solutions, Madison, WI, United States of America
| | - Amy J Weisman
- AIQ Solutions, Madison, WI, United States of America
| | - Robert Jeraj
- University of Wisconsin-Madison, Department of Medical Physics, Madison, WI, United States of America
- University of Ljubljana, Faculty of Mathematics and Physics, Ljubljana, SI, Slovenia
| | | |
Collapse
|