QIN Benchmarks for Clinical Translation of Quantitative Imaging Tools

A Quantitative Multiparametric MRI Analysis Platform for Estimation of Robust Imaging Biomarkers in Clinical Oncology

There is a need to develop user-friendly imaging tools estimating robust quantitative biomarkers (QIBs) from multiparametric (mp)MRI for clinical applications in oncology. Quantitative metrics derived from (mp)MRI can monitor and predict early responses to treatment, often prior to anatomical changes. We have developed a vendor-agnostic, flexible, and user-friendly MATLAB-based toolkit, MRI-Quantitative Analysis and Multiparametric Evaluation Routines ("MRI-QAMPER", current release v3.0), for the estimation of quantitative metrics from dynamic contrast-enhanced (DCE) and multi-b value diffusion-weighted (DW) MR and MR relaxometry. MRI-QAMPER's functionality includes generating numerical parametric maps from these methods reflecting tumor permeability, cellularity, and tissue morphology. MRI-QAMPER routines were validated using digital reference objects (DROs) for DCE and DW MRI, serving as initial approval stages in the National Cancer Institute Quantitative Imaging Network (NCI/QIN) software benchmark. MRI-QAMPER has participated in DCE and DW MRI Collaborative Challenge Projects (CCPs), which are key technical stages in the NCI/QIN benchmark. In a DCE CCP, QAMPER presented the best repeatability coefficient (RC = 0.56) across test-retest brain metastasis data, out of ten participating DCE software packages. In a DW CCP, QAMPER ranked among the top five (out of fourteen) tools with the highest area under the curve (AUC) for prostate cancer detection. This platform can seamlessly process mpMRI data from brain, head and neck, thyroid, prostate, pancreas, and bladder cancer. MRI-QAMPER prospectively analyzes dose de-escalation trial data for oropharyngeal cancer, which has earned it advanced NCI/QIN approval for expanded usage and applications in wider clinical trials.

A Review on the Use of Imaging Biomarkers in Oncology Clinical Trials: Quality Assurance Strategies for Technical Validation

Imaging biomarkers (IBs) have been proposed in medical literature that exploit images in a quantitative way, going beyond the visual assessment by an imaging physician. These IBs can be used in the diagnosis, prognosis, and response assessment of several pathologies and are very often used for patient management pathways. In this respect, IBs to be used in clinical practice and clinical trials have a requirement to be precise, accurate, and reproducible. Due to limitations in imaging technology, an error can be associated with their value when considering the entire imaging chain, from data acquisition to data reconstruction and subsequent analysis. From this point of view, the use of IBs in clinical trials requires a broadening of the concept of quality assurance and this can be a challenge for the responsible medical physics experts (MPEs). Within this manuscript, we describe the concept of an IB, examine some examples of IBs currently employed in clinical practice/clinical trials and analyze the procedure that should be carried out to achieve better accuracy and reproducibility in their use. We anticipate that this narrative review, written by the components of the EFOMP working group on "the role of the MPEs in clinical trials"-imaging sub-group, can represent a valid reference material for MPEs approaching the subject.

Post-acquisition standardization of positron emission tomography images

Purpose

Tissue radiotracer activity measured from positron emission tomography (PET) images is an important biomarker that is clinically utilized for diagnosis, staging, prognostication, and treatment response assessment in patients with cancer and other clinical disorders. Using PET image values to define a normal range of metabolic activity for quantification purposes is challenging due to variations in patient-related factors and technical factors. Although the formulation of standardized uptake value (SUV) has compensated for some of these variabilities, significant non-standardness still persists. We propose an image processing method to substantially mitigate these variabilities.

Methods

The standardization method is similar for activity concentration (AC) PET and SUV PET images with some differences and consists of two steps. The calibration step is performed only once for each of AC PET or SUV PET, employs a set of images of normal subjects, and requires a reference object, while the transformation step is executed for each patient image to be standardized. In the calibration step, a standardized scale is determined along with 3 key image intensity landmarks defined on it including the minimum percentile intensitys min , median intensitys m , and high percentile intensitys max . s min ands m are estimated based on image intensities within the body region in the normal calibration image set. The optimal value of the maximum percentile β corresponding to the intensitys max is estimated via an optimization process by using the reference object to optimally separate the highly variable high uptake values from the normal uptake intensities. In the transformation step, the first two landmarks-the minimum percentile intensityp α ( I ) , and the median intensityp m ( I ) -are found for the given image I for the body region, and the high percentile intensityp β ( I ) is determined corresponding to the optimally estimated high percentile value β . Subsequently, intensities of I are mapped to the standard scale piecewise linearly for different segments. We employ three strategies for evaluation and comparison with other standardization methods: (i) comparing coefficient of variationC V O of mean intensity within test objects O across different normal test subjects before and after standardization; (ii) comparing mean absolute difference (MD O ) of mean intensity within test objects O across different subjects in repeat scans before and after standardization; (iii) comparingC V O of mean intensity across different normal subjects before and after standardization where the scans came from different brands of scanners.

Results

Our data set consisted of 84 FDG-PET/CT scans of the body torso including 38 normal subjects and two repeat-scans of 23 patients. We utilized one of two objects-liver and spleen-as a reference object and the other for testing. The proposed standardization method reducedC V O andMD O by a factor of 3-8 in comparison to other standardization methods and no standardization. Upon standardization by our method, the image intensities (both for AC and SUV) from two different brands of scanners become statistically indistinguishable, while without standardization, they differ significantly and by a factor of 3-9.

Conclusions

The proposed method is automatic, outperforms current standardization methods, and effectively overcomes the residual variation left over in SUV and inter-scanner variations.

Post-acquisition Standardization of Positron Emission Tomography Images

Purpose

Tissue radiotracer activity measured from positron emission tomography (PET) images is an important biomarker that is clinically utilized for diagnosis, staging, prognostication, and treatment response assessment in patients with cancer and other clinical disorders. Using PET image values to define a normal range of metabolic activity for quantification purposes is challenging due to variations in patient-related factors and technical factors. Although the formulation of standardized uptake value (SUV) has compensated for some of these variabilities, significant non-standardness still persists. We propose an image processing method to substantially mitigate these variabilities.

Methods

The standardization method is similar for activity concentration (AC) PET and SUV PET images with some differences and consists of two steps. The calibration step is performed only once for each of AC PET or SUV PET, employs a set of images of normal subjects, and requires a reference object, while the transformation step is executed for each patient image to be standardized. In the calibration step, a standardized scale is determined along with 3 key image intensity landmarks defined on it including the minimum percentile intensity smin, median intensity sm, and high percentile intensity smax. smin and sm are estimated based on image intensities within the body region in the normal calibration image set. The optimal value of the maximum percentile β corresponding to the intensity smax is estimated via an optimization process by using the reference object to optimally separate the highly variable high uptake values from the normal uptake intensities. In the transformation step, the first two landmarks - the minimum percentile intensity pα(I), and the median intensity pm(I) - are found for the given image I for the body region, and the high percentile intensity pβ(I) is determined corresponding to the optimally estimated high percentile value β. Subsequently, intensities of I are mapped to the standard scale piecewise linearly for different segments.We employ three strategies for evaluation and comparison with other standardization methods: (i) Comparing coefficient of variation (CVO) of mean intensity within test objects O across different normal test subjects before and after standardization; (ii) Comparing mean absolute difference (MDO) of mean intensity within test objects O across different subjects in repeat scans before and after standardization; (iii) Comparing CVO of mean intensity across different normal subjects before and after standardization where the scans came from different brands of scanners.

Results

Our data set consisted of 84 FDG-PET/CT scans of the body torso including 38 normal subjects and two repeat-scans of 23 patients. We utilized one of two objects - liver and spleen - as a reference object and the other for testing. The proposed standardization method reduced CVO and MDO by a factor of 3-8 in comparison to other standardization methods and no standardization. Upon standardization by our method, the image intensities (both for AC and SUV) from two different brands of scanners become statistically indistinguishable, while without standardization, they differ significantly and by a factor of 3-9.

Conclusions

The proposed method is automatic, outperforms current standardization methods, and effectively overcomes the residual variation left over in SUV and inter-scanner variations.

In vivo tumor immune microenvironment phenotypes correlate with inflammation and vasculature to predict immunotherapy response

Response to immunotherapies can be variable and unpredictable. Pathology-based phenotyping of tumors into ‘hot’ and ‘cold’ is static, relying solely on T-cell infiltration in single-time single-site biopsies, resulting in suboptimal treatment response prediction. Dynamic vascular events (tumor angiogenesis, leukocyte trafficking) within tumor immune microenvironment (TiME) also influence anti-tumor immunity and treatment response. Here, we report dynamic cellular-level TiME phenotyping in vivo that combines inflammation profiles with vascular features through non-invasive reflectance confocal microscopic imaging. In skin cancer patients, we demonstrate three main TiME phenotypes that correlate with gene and protein expression, and response to toll-like receptor agonist immune-therapy. Notably, phenotypes with high inflammation associate with immunostimulatory signatures and those with high vasculature with angiogenic and endothelial anergy signatures. Moreover, phenotypes with high inflammation and low vasculature demonstrate the best treatment response. This non-invasive in vivo phenotyping approach integrating dynamic vasculature with inflammation serves as a reliable predictor of response to topical immune-therapy in patients.

Standard assessment of immune infiltration of biopsies is not sufficient to accurately predict response to immunotherapy. Here, the authors show that reflectance confocal microscopy can be used to quantify dynamic vasculature and inflammatory features to better predict treatment response in skin cancers.

Parametric Response Mapping of FLAIR MRI Provides an Early Indication of Progression Risk in Glioblastoma

RATIONALE AND OBJECTIVES

Glioblastoma image evaluation utilizes Magnetic Resonance Imaging contrast-enhanced, T1-weighted, and noncontrast T2-weighted fluid-attenuated inversion recovery (FLAIR) acquisitions. Disease progression assessment relies on changes in tumor diameter, which correlate poorly with survival. To improve treatment monitoring in glioblastoma, we investigated serial voxel-wise comparison of anatomically-aligned FLAIR signal as an early predictor of GBM progression.

MATERIALS AND METHODS

We analyzed longitudinal normalized FLAIR images (rFLAIR) from 52 subjects using voxel-wise Parametric Response Mapping (PRM) to monitor volume fractions of increased (PRM_rFLAIR+), decreased (PRM_rFLAIR-), or unchanged (PRM_rFLAIR0) rFLAIR intensity. We determined response by rFLAIR between pretreatment and 10 weeks posttreatment. Risk of disease progression in a subset of subjects (N = 26) with stable disease or partial response as defined by Response Assessment in Neuro-Oncology (RANO) criteria was assessed by PRM_rFLAIR between weeks 10 and 20 and continuously until the PRM_rFLAIR+ exceeded a defined threshold. RANO defined criteria were compared with PRM-derived outcomes for tumor progression detection.

RESULTS

Patient stratification for progression-free survival (PFS) and overall survival (OS) was achieved at week 10 using RANO criteria (PFS: p <0.0001; OS: p <0.0001), relative change in FLAIR-hyperintense volume (PFS: p = 0.0011; OS: p <0.0001), and PRM_rFLAIR+ (PFS: p <0.01; OS: p <0.001). PRM_rFLAIR+ also stratified responding patients' progression between weeks 10 and 20 (PFS: p <0.05; OS: p = 0.01) while changes in FLAIR-volume measurements were not predictive. As a continuous evaluation, PRM_rFLAIR+ exceeding 10% stratified patients for PFA after 5.6 months (p<0.0001), while RANO criteria did not stratify patients until 15.4 months (p <0.0001).

CONCLUSION

PRM_rFLAIR may provide an early biomarker of disease progression in glioblastoma.

Development, validation, qualification, and dissemination of quantitative MR methods: Overview and recommendations by the ISMRM quantitative MR study group

On behalf of the International Society for Magnetic Resonance in Medicine (ISMRM) Quantitative MR Study Group, this article provides an overview of considerations for the development, validation, qualification, and dissemination of quantitative MR (qMR) methods. This process is framed in terms of two central technical performance properties, i.e., bias and precision. Although qMR is confounded by undesired effects, methods with low bias and high precision can be iteratively developed and validated. For illustration, two distinct qMR methods are discussed throughout the manuscript: quantification of liver proton-density fat fraction, and cardiac T₁ . These examples demonstrate the expansion of qMR methods from research centers toward widespread clinical dissemination. The overall goal of this article is to provide trainees, researchers, and clinicians with essential guidelines for the development and validation of qMR methods, as well as an understanding of necessary steps and potential pitfalls for the dissemination of quantitative MR in research and in the clinic.

Reliability of Quantitative 18F-FDG PET/CT Imaging Biomarkers for Classifying Early Response to Chemoradiotherapy in Patients With Locally Advanced Non-Small Cell Lung Cancer

PURPOSE OF THE REPORT

We evaluated the reliability of 18F-FDG PET imaging biomarkers to classify early response status across observers, scanners, and reconstruction algorithms in support of biologically adaptive radiation therapy for locally advanced non-small cell lung cancer.

PATIENTS AND METHODS

Thirty-one patients with unresectable locally advanced non-small cell lung cancer were prospectively enrolled on a phase 2 trial (NCT02773238) and underwent 18F-FDG PET on GE Discovery STE (DSTE) or GE Discovery MI (DMI) PET/CT systems at baseline and during the third week external beam radiation therapy regimens. All PET scans were reconstructed using OSEM; GE-DMI scans were also reconstructed with BSREM-TOF (block sequential regularized expectation maximization reconstruction algorithm incorporating time of flight). Primary tumors were contoured by 3 observers using semiautomatic gradient-based segmentation. SUVmax, SUVmean, SUVpeak, MTV (metabolic tumor volume), and total lesion glycolysis were correlated with midtherapy multidisciplinary clinical response assessment. Dice similarity of contours and response classification areas under the curve were evaluated across observers, scanners, and reconstruction algorithms. LASSO logistic regression models were trained on DSTE PET patient data and independently tested on DMI PET patient data.

RESULTS

Interobserver variability of PET contours was low for both OSEM and BSREM-TOF reconstructions; intraobserver variability between reconstructions was slightly higher. ΔSUVpeak was the most robust response predictor across observers and image reconstructions. LASSO models consistently selected ΔSUVpeak and ΔMTV as response predictors. Response classification models achieved high cross-validated performance on the DSTE cohort and more variable testing performance on the DMI cohort.

CONCLUSIONS

The variability FDG PET lesion contours and imaging biomarkers was relatively low across observers, scanners, and reconstructions. Objective midtreatment PET response assessment may lead to improved precision of biologically adaptive radiation therapy.

Comparison of Segmentation Methods in Assessing Background Parenchymal Enhancement as a Biomarker for Response to Neoadjuvant Therapy

Breast parenchymal enhancement (BPE) has shown association with breast cancer risk and response to neoadjuvant treatment. However, BPE quantification is challenging, and there is no standardized segmentation method for measurement. We investigated the use of a fully automated breast fibroglandular tissue segmentation method to calculate BPE from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) for use as a predictor of pathologic complete response (pCR) following neoadjuvant treatment in the I-SPY 2 TRIAL. In this trial, patients had DCE-MRI at baseline (T0), after 3 weeks of treatment (T1), after 12 weeks of treatment and between drug regimens (T2), and after completion of treatment (T3). A retrospective analysis of 2 cohorts was performed: one with 735 patients and another with a final cohort of 340 patients, meeting a high-quality benchmark for segmentation. We evaluated 3 subvolumes of interest segmented from bilateral T1-weighted axial breast DCE-MRI: full stack (all axial slices), half stack (center 50% of slices), and center 5 slices. The differences between methods were assessed, and a univariate logistic regression model was implemented to determine the predictive performance of each segmentation method. The results showed that the half stack method provided the best compromise between sampling error from too little tissue and inclusion of incorrectly segmented tissues from extreme superior and inferior regions. Our results indicate that BPE calculated using the half stack segmentation approach has potential as an early biomarker for response to treatment in the hormone receptor–negative and human epidermal growth factor receptor 2–positive subtype.

Repeatability and Reproducibility of ADC Histogram Metrics from the ACRIN 6698 Breast Cancer Therapy Response Trial

Mean tumor apparent diffusion coefficient (ADC) of breast cancer showed excellent repeatability but only moderate predictive power for breast cancer therapy response in the ACRIN 6698 multicenter imaging trial. Previous single-center studies have shown improved predictive performance for alternative ADC histogram metrics related to low ADC dense tumor volume. Using test/retest (TT/RT) 4 b-value diffusion-weighted imaging acquisitions from pretreatment or early-treatment time-points on 71 ACRIN 6698 patients, we evaluated repeatability for ADC histogram metrics to establish confidence intervals and inform predictive models for future therapy response analysis. Histograms were generated using regions of interest (ROIs) defined separately for TT and RT diffusion-weighted imaging. TT/RT repeatability and intra- and inter-reader reproducibility (on a 20-patient subset) were evaluated using wCV and Bland–Altman limits of agreement for histogram percentiles, low-ADC dense tumor volumes, and fractional volumes (normalized to total histogram volume). Pearson correlation was used to reveal connections between metrics and ROI variability across the sample cohort. Low percentiles (15th and 25th) were highly repeatable and reproducible, wCV < 8.1%, comparable to mean ADC values previously reported. Volumetric metrics had higher wCV values in all cases, with fractional volumes somewhat better but at least 3 times higher than percentile wCVs. These metrics appear most sensitive to ADC changes around a threshold of 1.2 μm²/ms. Volumetric results were moderately to strongly correlated with ROI size. In conclusion, Lower histogram percentiles have comparable repeatability to mean ADC, while ADC-thresholded volumetric measures currently have poor repeatability but may benefit from improvements in ROI techniques.

Radiogenomics of Gliomas

The 2016 World Health Organization brain tumor classification is based on genomic and molecular profile of tumor tissue. These characteristics have improved understanding of the brain tumor and played an important role in treatment planning and prognostication. There is an ongoing effort to develop noninvasive imaging techniques that provide insight into tissue characteristics at the cellular and molecular levels. This article focuses on the molecular characteristics of gliomas, transcriptomic subtypes, and radiogenomic studies using semantic and radiomic features. The limitations and future directions of radiogenomics as a standalone diagnostic tool also are discussed.

Quantitative Imaging Enters the Clinical Arena: A Personal Viewpoint

The National Cancer Institute's Quantitative Imaging Network (QIN) has thrived over the past 12 years with an emphasis on the development of image-based decision support software tools for improving measurements of imaging metrics. An overarching goal has been to develop advanced tools that could be translated into clinical trials to provide for improved prediction of response to therapeutic interventions. This article provides an overview of the successes in development and translation of new algorithms into the clinical workflow by the many research teams of the Quantitative Imaging Network.

Clinical Trial Design and Development Work Group Within the Quantitative Imaging Network

The Clinical Trial Design and Development Working Group within the Quantitative Imaging Network focuses on providing support for the development, validation, and harmonization of quantitative imaging (QI) methods and tools for use in cancer clinical trials. In the past 10 years, the Group has been working in several areas to identify challenges and opportunities in clinical trials involving QI and radiation oncology. The Group has been working with Quantitative Imaging Network members and the Quantitative Imaging Biomarkers Alliance leadership to develop guidelines for standardizing the reporting of quantitative imaging. As a validation platform, the Group led a multireader study to test a semi-automated positron emission tomography quantification software. Clinical translation of QI tools cannot be possible without a continuing dialogue with clinical users. This article also highlights the outreach activities extended to cooperative groups and other organizations that promote the use of QI tools to support clinical decisions.

Quantitative Imaging Informatics for Cancer Research

PURPOSE

We summarize Quantitative Imaging Informatics for Cancer Research (QIICR; U24 CA180918), one of the first projects funded by the National Cancer Institute (NCI) Informatics Technology for Cancer Research program.

METHODS

QIICR was motivated by the 3 use cases from the NCI Quantitative Imaging Network. 3D Slicer was selected as the platform for implementation of open-source quantitative imaging (QI) tools. Digital Imaging and Communications in Medicine (DICOM) was chosen for standardization of QI analysis outputs. Support of improved integration with community repositories focused on The Cancer Imaging Archive (TCIA). Priorities included improved capabilities of the standard, toolkits and tools, reference datasets, collaborations, and training and outreach.

RESULTS

Fourteen new tools to support head and neck cancer, glioblastoma, and prostate cancer QI research were introduced and downloaded over 100,000 times. DICOM was amended, with over 40 correction proposals addressing QI needs. Reference implementations of the standard in a popular toolkit and standalone tools were introduced. Eight datasets exemplifying the application of the standard and tools were contributed. An open demonstration/connectathon was organized, attracting the participation of academic groups and commercial vendors. Integration of tools with TCIA was improved by implementing programmatic communication interface and by refining best practices for QI analysis results curation.

CONCLUSION

Tools, capabilities of the DICOM standard, and datasets we introduced found adoption and utility within the cancer imaging community. A collaborative approach is critical to addressing challenges in imaging informatics at the national and international levels. Numerous challenges remain in establishing and maintaining the infrastructure of analysis tools and standardized datasets for the imaging community. Ideas and technology developed by the QIICR project are contributing to the NCI Imaging Data Commons currently being developed.