Quantitative imaging biomarkers: a review of statistical methods for technical performance assessment

Systematic data analysis pipeline for quantitative morphological cell phenotyping

Quantitative morphological phenotyping (QMP) is an image-based method used to capture morphological features at both the cellular and population level. Its interdisciplinary nature, spanning from data collection to result analysis and interpretation, can lead to uncertainties, particularly among those new to this actively growing field. High analytical specificity for a typical QMP is achieved through sophisticated approaches that can leverage subtle cellular morphological changes. Here, we outline a systematic workflow to refine the QMP methodology. For a practical review, we describe the main steps of a typical QMP; in each step, we discuss the available methods, their applications, advantages, and disadvantages, along with the R functions and packages for easy implementation. This review does not cover theoretical backgrounds, but provides several references for interested researchers. It aims to broaden the horizons for future phenome studies and demonstrate how to exploit years of endeavors to achieve more with less.

Test-retest repeatability of myocardial radiomic features from quantitative cardiac magnetic resonance T1 and T2 mapping

Radiomics of cardiac magnetic resonance (MR) imaging has proved to be potentially useful in the study of various myocardial diseases. Therefore, assessing the repeatability degree in radiomic features measurement is of fundamental importance. The aim of this study was to assess test-retest repeatability of myocardial radiomic features extracted from quantitative T1 and T2 maps. A representative group of 24 subjects (mean age 54 ± 18 years) referred for clinical cardiac MR imaging were enrolled in the study. For each subject, T1 and T2 mapping through MOLLI and T2-prepared TrueFISP acquisition sequences, respectively, were performed at 1.5 T. Then, 98 radiomic features of different classes (shape, first-order, second-order) were extracted from a region of interest encompassing the whole left ventricle myocardium in a short axis slice. The repeatability was assessed performing different and complementary analyses: intraclass correlation coefficient (ICC) and limits of agreement (LOA) (i.e., the interval within which 95% of the percentage differences between two repeated measures are expected to lie). Radiomic features were characterized by a relatively wide range of repeatability degree in terms of both ICC and LOA. Overall, 44.9% and 38.8% of radiomic features showed ICC values > 0.75 for T1 and T2 maps, respectively, while 25.5% and 23.4% of radiomic features showed LOA between ±10%. A subset of radiomic features for T1 (Mean, Median, 10Percentile, 90Percentile, RootMeanSquared, Imc2, RunLengthNonUniformityNormalized, RunPercentage and ShortRunEmphasis) and T2 (MaximumDiameter, RunLengthNonUniformityNormalized, RunPercentage, ShortRunEmphasis) maps presented both ICC > 0.75 and LOA between ±5%. Overall, radiomic features extracted from T1 maps showed better repeatability performance than those extracted from T2 maps, with shape features characterized by better repeatability than first-order and textural features. Moreover, only a limited subset of 9 and 4 radiomic features for T1 and T2 maps, respectively, showed high repeatability degree in terms of both ICC and LOA. These results confirm the importance of assessing test-retest repeatability degree in radiomic feature estimation and might be useful for a more effective/reliable use of myocardial T1 and T2 mapping radiomics in clinical or research studies.

Normal Brain and Brain Tumor ADC: Changes Resulting From Variation of Diffusion Time and/or Echo Time in Pulsed-Gradient Spin Echo Diffusion Imaging

OBJECTIVES

Increasing gradient performance on modern magnetic resonance imaging scanners has profoundly reduced the attainable diffusion and echo times for clinically available pulsed-gradient spin echo (PGSE) sequences. This study investigated how this may impact the measured apparent diffusion coefficient (ADC), which is considered an important diagnostic marker for differentiation between normal and abnormal brain tissue and for therapeutic follow-up.

MATERIALS AND METHODS

Diffusion time and echo time dependence of the ADC were evaluated on a high-performance 3 T magnetic resonance imaging scanner. Diffusion PGSE brain scans were performed in 10 healthy volunteers and in 10 brain tumor patients using diffusion times of 16, 40, and 70 ms, echo times of 60, 75, and 104 ms at 3 b-values (0, 100, and 1000 s/mm 2 ), and a maximum gradient amplitude of 68 mT/m. A low gradient performance system was also emulated by reducing the diffusion encoding gradient amplitude to 19 mT/m. In healthy subjects, the ADC was measured in 6 deep gray matter regions and in 6 white matter regions. In patients, the ADC was measured in the solid part of the tumor.

RESULTS

With increasing diffusion time, a small but significant ADC increase of up to 2.5% was observed for 6 aggregate deep gray matter structures. With increasing echo time or reduced gradient performance, a small but significant ADC decrease of up to 2.6% was observed for 6 aggregate white matter structures. In tumors, diffusion time-related ADC changes were inconsistent without clear trend. For tumors with diffusivity above 1.0 μm 2 /ms, with prolonged echo time, there was a pronounced ADC increase of up to 12%. Meanwhile, for tumors with diffusivity at or below 1.0 μm 2 /ms, no change or a reduction was observed. Similar results were observed for gradient performance reduction, with an increase of up to 21%. The coefficient of variation determined in repeat experiments was 2.4%.

CONCLUSIONS

For PGSE and the explored parameter range, normal tissue ADC changes seem negligible. Meanwhile, observed tumor ADC changes can be relevant if ADC is used as a quantitative biomarker and not merely assessed by visual inspection. This highlights the importance of reporting all pertinent timing parameters in ADC studies and of considering these effects when building scan protocols for use in multicenter investigations.

Three-dimensional simultaneous T1 and T2* relaxation times and quantitative susceptibility mapping at 3 T: A multicenter validation study

We aimed to determine the intra-site repeatability and cross-site reproducibility of T1 and T2* relaxation times and quantitative susceptibility (χ) values obtained through quantitative parameter mapping (QPM) at 3 T. This prospective study included three 3-T scanners with the same hardware and software platform at three sites. The brains of twelve healthy volunteers were scanned three times using QPM at three sites. Intra-site repeatability and cross-site reproducibility were evaluated based on voxel-wise and region-of-interest analyses. The within-subject coefficient of variation (wCV), within-subject standard deviation (wSD), linear regression, Bland-Altman plot, and intraclass correlation coefficient (ICC) were used for evaluation. The intra-site repeatability wCV was 11.9 ± 6.86% for T1 and 3.15 ± 0.03% for T2*, and wSD of χ at 3.35 ± 0.10 parts per billion (ppb). Intra-site ICC(1,k) values for T1, T2*, and χ were 0.878-0.904, 0.972-0.976, and 0.966-0.972, respectively, indicating high consistency within the same scanner. Linear regression analysis revealed a strong agreement between measurements from each site and the site-average measurement, with R-squared values ranging from 0.79 to 0.83 for T1, 0.94-0.95 for T2*, and 0.95-0.96 for χ. The cross-site wCV was 13.4 ± 5.47% for T1 and 3.69 ± 2.25% for T2*, and cross-site wSD of χ at 4.08 ± 3.22 ppb. The cross-site ICC(2,1) was 0.707, 0.913, and 0.902 for T1, T2*, and χ, respectively. QPM provides T1, T2*, and χ values with an intra-site repeatability of <12% and cross-site reproducibility of <14%. These findings may contribute to the development of multisite studies.

Nonparametric Biomarker Based Treatment Selection With Reproducibility Data

We consider evaluating biomarkers for treatment selection under assay modification. Survival outcome, treatment, and Affymetrix gene expression data were attained from cancer patients. Consider migrating a gene expression biomarker to the Illumina platform. A recent novel approach allows a quick evaluation of the migrated biomarker with only a reproducibility study needed to compare the two platforms, achieved by treating the original biomarker as an error-contaminated observation of the migrated biomarker. However, its assumptions of a classical measurement error model and a linear predictor for the outcome may not hold. Ignoring such model deviations may lead to sub-optimal treatment selection or failure to identify effective biomarkers. To overcome such limitations, we adopt a nonparametric logistic regression to model the relationship between the event rate and the biomarker, and the deduced marker-based treatment selection is optimal. We further assume a nonparametric relationship between the migrated and original biomarkers and show that the error-contaminated biomarker leads to sub-optimal treatment selection compared to the error-free biomarker. We obtain the estimation via B-spline approximation. The approach is assessed by simulation studies and demonstrated through application to lung cancer data.

Multicenter, multivendor validation of liver quantitative susceptibility mapping in patients with iron overload at 1.5 T and 3 T

PURPOSE

To evaluate the repeatability and reproducibility of QSM of the liver via single breath-hold chemical shift-encoded MRI at both 1.5 T and 3 T in a multicenter, multivendor study in subjects with iron overload.

METHODS

This prospective study included four academic medical centers with three different MRI vendors at 1.5 T and 3 T. Subjects with known or suspected liver iron overload underwent multi-echo spoiled gradient-recalled-echo scans at each field strength. A subset received repeatability testing at either 1.5 T or 3 T. Susceptibility andR 2 * $$ {\mathrm{R}}_2^{\ast } $$ maps were reconstructed from the multi-echo images and analyzed at a single center. QSM-measured susceptibility was compared withR 2 * $$ {\mathrm{R}}_2^{\ast } $$ and a commercial R2-based liver iron concentration method across centers and field strengths using linear regression and F-tests on the intercept and slope. Field-strength reproducibility and test/retest repeatability were evaluated using Bland-Altman analysis.

RESULTS

A total of 155/80 data sets (test/retest) were available at 1.5 T, and 159/70 data sets (test/retest) were available at 3 T. Calibrations across sites were reproducible, with some variability (e.g., susceptibility slope with liver iron concentration ranged from 0.102 to 0.123 g/[mg· $$ \cdotp $$ ppm] across centers at 1.5 T). Field strength reproducibility was good (concordance correlation coefficient = 0.862), and test/retest repeatability was excellent (intraclass correlation coefficient = 0.951).

CONCLUSION

QSM as an imaging biomarker of liver iron overload is feasible and repeatable across centers and MR vendors. It may be complementary withR 2 * $$ {\mathrm{R}}_2^{\ast } $$ as they are obtained from the same acquisition. Although good reproducibility was observed, liver QSM may benefit from standardization of acquisition parameters. Overall, QSM is a promising method for liver iron quantification.

Conceptual review of outcome metrics and measures used in clinical evaluation of artificial intelligence in radiology

Artificial intelligence (AI) has numerous applications in radiology. Clinical research studies to evaluate the AI models are also diverse. Consequently, diverse outcome metrics and measures are employed in the clinical evaluation of AI, presenting a challenge for clinical radiologists. This review aims to provide conceptually intuitive explanations of the outcome metrics and measures that are most frequently used in clinical research, specifically tailored for clinicians. While we briefly discuss performance metrics for AI models in binary classification, detection, or segmentation tasks, our primary focus is on less frequently addressed topics in published literature. These include metrics and measures for evaluating multiclass classification; those for evaluating generative AI models, such as models used in image generation or modification and large language models; and outcome measures beyond performance metrics, including patient-centered outcome measures. Our explanations aim to guide clinicians in the appropriate use of these metrics and measures.

Reproducibility and Repeatability of US Shear-Wave and Transient Elastography in Nonalcoholic Fatty Liver Disease

Background US shear-wave elastography (SWE) and vibration-controlled transient elastography (VCTE) enable assessment of liver stiffness, an indicator of fibrosis severity. However, limited reproducibility data restrict their use in clinical trials. Purpose To estimate SWE and VCTE measurement variability in nonalcoholic fatty liver disease (NAFLD) within and across systems to support clinical trial diagnostic enrichment and clinical interpretation of longitudinal liver stiffness. Materials and Methods This prospective, observational, cross-sectional study (March 2021 to November 2021) enrolled adults with NAFLD, stratified according to the Fibrosis-4 (FIB-4) index (≤1.3, >1.3 and <2.67, ≥2.67), at two sites to assess SWE with five US systems and VCTE with one system. Each participant underwent 12 elastography examinations over two separate days within 1 week, with each day's examinations conducted by a different operator. VCTE and SWE measurements were reported in units of meters per second. The primary end point was the different-day, different-operator reproducibility coefficient (RDCDDDO) pooled across systems for SWE and individually for VCTE. Secondary end points included system-specific RDCDDDO, same-day, same-operator repeatability coefficient (RCSDSO), and between-system same-day, same-operator reproducibility coefficient. The planned sample provided 80% power to detect a pooled RDCDDDO of less than 35%, the prespecified performance threshold. Results A total of 40 participants (mean age, 60 years ± 10 [SD]; 24 women) with low (n = 17), intermediate (n = 15), and high (n = 8) FIB-4 scores were enrolled. RDCDDDO was 30.7% (95% upper bound, 34.4%) for SWE and 35.6% (95% upper bound, 43.9%) for VCTE. SWE system-specific RDCDDDO varied from 24.2% to 34.3%. The RCSDSO was 21.0% for SWE (range, 13.9%-35.0%) and 19.6% for VCTE. The SWE between-system same-day, same-operator reproducibility coefficient was 52.7%. Conclusion SWE met the prespecified threshold, RDCDDDO less than 35%, with VCTE having a higher RDCDDDO. SWE variability was higher between different systems. These estimates advance liver US-based noninvasive test qualification by (a) defining expected variability, (b) establishing that serial examination variability is lower when performed with the same system, and (c) informing clinical trial design. ClinicalTrials.gov Identifier NCT04828551 © RSNA, 2024 Supplemental material is available for this article.

Measuring repeatability of dynamic contrast-enhanced MRI biomarkers improves evaluation of biological response to radiotherapy in lung cancer

OBJECTIVES

To measure dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) biomarker repeatability in patients with non-small cell lung cancer (NSCLC). To use these statistics to identify which individual target lesions show early biological response.

MATERIALS AND METHODS

A single-centre, prospective DCE-MRI study was performed between September 2015 and April 2017. Patients with NSCLC were scanned before standard-of-care radiotherapy to evaluate biomarker repeatability and two weeks into therapy to evaluate biological response. Volume transfer constant (Ktrans), extravascular extracellular space volume fraction (ve) and plasma volume fraction (vp) were measured at each timepoint along with tumour volume. Repeatability was assessed using a within-subject coefficient of variation (wCV) and repeatability coefficient (RC). Cohort treatment effects on biomarkers were estimated using mixed-effects models. RC limits of agreement revealed which individual target lesions changed beyond that expected with biomarker daily variation.

RESULTS

Fourteen patients (mean age, 67 years +/- 12, 8 men) had 22 evaluable lesions (12 primary tumours, 8 nodal metastases, 2 distant metastases). The wCV (in 8/14 patients) was between 9.16% to 17.02% for all biomarkers except for vp, which was 42.44%. Cohort-level changes were significant for Ktrans and ve (p < 0.001) and tumour volume (p = 0.002). Ktrans and tumour volume consistently showed the greatest number of individual lesions showing biological response. In distinction, no individual lesions had a real change in ve despite the cohort-level change.

CONCLUSION

Identifying individual early biological responders provided additional information to that derived from conventional cohort cohort-level statistics, helping to prioritise which parameters would be best taken forward into future studies.

CLINICAL RELEVANCE STATEMENT

Dynamic contrast-enhanced magnetic resonance imaging biomarkers Ktrans and tumour volume are repeatable and detect early treatment-induced changes at both cohort and individual lesion levels, supporting their use in further evaluation of radiotherapy and targeted therapeutics.

KEY POINTS

Few literature studies report quantitative imaging biomarker precision, by measuring repeatability or reproducibility. Several DCE-MRI biomarkers of lung cancer tumour microenvironment were highly repeatable. Repeatability coefficient measurements enabled lesion-specific evaluation of early biological response to therapy, improving conventional assessment.

Phantoms for Quantitative Body MRI: a review and discussion of the phantom value

In this paper, we review the value of phantoms for body MRI in the context of their uses for quantitative MRI methods research, clinical trials, and clinical imaging. Certain uses of phantoms are common throughout the body MRI community, including measuring bias, assessing reproducibility, and training. In addition to these uses, phantoms in body MRI methods research are used for novel methods development and the design of motion compensation and mitigation techniques. For clinical trials, phantoms are an essential part of quality management strategies, facilitating the conduct of ethically sound, reliable, and regulatorily compliant clinical research of both novel MRI methods and therapeutic agents. In the clinic, phantoms are used for development of protocols, mitigation of cost, quality control, and radiotherapy. We briefly review phantoms developed for quantitative body MRI, and finally, we review open questions regarding the most effective use of a phantom for body MRI.

Assessment of hepatic transporter function in rats using dynamic gadoxetate-enhanced MRI: a reproducibility study

OBJECTIVE

Previous studies have revealed a substantial between-centre variability in DCE-MRI biomarkers of hepatocellular function in rats. This study aims to identify the main sources of variability by comparing data measured at different centres and field strengths, at different days in the same subjects, and over the course of several months in the same centre.

MATERIALS AND METHODS

13 substudies were conducted across three facilities on two 4.7 T and two 7 T scanners using a 3D spoiled gradient echo acquisition. All substudies included 3-6 male Wistar-Han rats each, either scanned once with vehicle (n = 76) or twice with either vehicle (n = 19) or 10 mg/kg of rifampicin (n = 13) at follow-up. Absolute values, between-centre reproducibility, within-subject repeatability, detection limits, and effect sizes were derived for hepatocellular uptake rate (Ktrans) and biliary excretion rate (kbh). Sources of variability were identified using analysis of variance and stratification by centre, field strength, and time period.

RESULTS

Data showed significant differences between substudies of 31% for Ktrans (p = 0.013) and 43% for kbh (p < 0.001). Within-subject differences were substantially smaller for kbh (8%) but less so for Ktrans (25%). Rifampicin-induced inhibition was safely above the detection limits, with an effect size of 75 ± 3% in Ktrans and 67 ± 8% in kbh. Most of the variability in individual data was accounted for by between-subject (Ktrans = 23.5%; kbh = 42.5%) and between-centre (Ktrans = 44.9%; kbh = 50.9%) variability, substantially more than the between-day variation (Ktrans = 0.1%; kbh = 5.6%). Significant differences in kbh were found between field strengths at the same centre, between centres at the same field strength, and between repeat experiments over 2 months apart in the same centre.

DISCUSSION

Between-centre bias caused by factors such as hardware differences, subject preparations, and operator dependence is the main source of variability in DCE-MRI of liver function in rats, closely followed by biological between-subject differences. Future method development should focus on reducing these sources of error to minimise the sample sizes needed to detect more subtle levels of inhibition.

Special Report on the Consensus QIBA Profile for Objective Analytical Validation of Non-calcified and High-risk Plaque and Other Biomarkers using Computed Tomography Angiography

RATIONALE AND OBJECTIVES

Evidence is building in support of the clinical utility of atherosclerotic plaque imaging by computed tomography angiography (CTA). There is increasing organized activity to embrace non-calcified plaque (NCP) as a formally defined biomarker for clinical trials, and high-risk plaque (HRP) for clinical care, as the most relevant measures for the field to advance and worthy of community efforts to validate. Yet the ability to assess the quantitative performance of any given specific solution to make these measurements or classifications is not available. Vendors use differing definitions, assessment metrics, and validation data sets to describe their offerings without clinician users having the capability to make objective assessments of accuracy and precision and how this affects diagnostic confidence.

MATERIALS AND METHODS

The QIBA Profile for Atherosclerosis Biomarkers by CTA was created by the Quantitative Imaging Biomarkers Alliance (QIBA) to improve objectivity and decrease the variability of noninvasive plaque phenotyping. The Profile provides claims on the accuracy and precision of plaque measures individually and when combined.

RESULTS

Individual plaque morphology measurements are evaluated in terms of bias (accuracy), slope (consistency of the bias across the measurement range, needed for measurements of change), and variability. The multiparametric plaque stability phenotype is evaluated in terms of agreement with expert pathologists. The Profile is intended for a broad audience, including those engaged in discovery science, clinical trials, and patient care.

CONCLUSION

This report provides a rationale and overview of the Profile claims and how to comply with the Profile in research and clinical practice.

SUMMARY STATEMENT

This article summarizes objective means to validate the analytical performance of non-calcified plaque (NCP), other emerging plaque morphology measurements, and multiparametric histology-defined high-risk plaque (HRP), as outlined in the QIBA Profile for Atherosclerosis Biomarkers by CTA.

Self-Gated Radial Free-Breathing Liver MR Elastography: Assessment of Technical Performance in Children at 3 T

BACKGROUND

Conventional liver magnetic resonance elastography (MRE) requires breath-holding (BH) to avoid motion artifacts, which is challenging for children. While radial free-breathing (FB)-MRE is an alternative for quantifying liver stiffness (LS), previous methods had limitations of long scan times, acquiring two slices in 5 minutes, and not resolving motion during reconstruction.

PURPOSE

To reduce FB-MRE scan time to 4 minutes for four slices and to investigate the impact of self-gated (SG) motion compensation on FB-MRE LS quantification in terms of agreement, intrasession repeatability, and technical quality compared to conventional BH-MRE.

STUDY TYPE

Prospective.

POPULATION

Twenty-six children without fibrosis (median age: 12.9 years, 15 females).

FIELD STRENGTH/SEQUENCE

3 T; Cartesian gradient-echo (GRE) BH-MRE, research application radial GRE FB-MRE.

ASSESSMENT

Participants were scanned twice to measure repeatability, without moving the table or changing the participants' position. LS was measured in areas of the liver with numerical confidence ≥90%. Technical quality was examined using measurable liver area (%).

STATISTICAL TESTS

Agreement of LS between BH-MRE and FB-MRE was evaluated using Bland-Altman analysis for SG acceptance rates of 40%, 60%, 80%, and 100%. LS repeatability was assessed using within-subject coefficient of variation (wCV). The differences in LS and measurable liver area were examined using Kruskal-Wallis and Wilcoxon signed-rank tests. P < 0.05 was considered significant.

RESULTS

FB-MRE with 60% SG achieved the closest agreement with BH-MRE (mean difference 0.00 kPa). The LS ranged from 1.70 to 1.83 kPa with no significant differences between BH-MRE and FB-MRE with varying SG rates (P = 0.52). All tested methods produced repeatable LS with wCV from 4.4% to 6.5%. The median measurable liver area was smaller for FB-MRE (32%-45%) than that for BH-MRE (91%-93%) (P < 0.05).

DATA CONCLUSION

FB-MRE with 60% SG can quantify LS with close agreement and comparable repeatability with respect to BH-MRE in children.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: Stage 1.

Quantitative Magnetic Resonance Imaging (qMRI) of the Small Bowel in Crohn's Disease: State-of-the-Art and Future Directions

Crohn's disease (CD) is a chronic inflammatory disease of the gastrointestinal tract in which repeated episodes of acute inflammation may lead to long-term bowel damage. Cross-sectional imaging is used in conjunction with endoscopy to diagnose and monitor disease and detect complications. Magnetic resonance imaging (MRI) has demonstrable utility in evaluating inflammatory activity. However, subjective interpretation of conventional MR sequences is limited in its ability to fully phenotype the underlying histopathological processes in chronic disease. In particular, conventional MRI can be confounded by the presence of mural fibrosis and muscle hypertrophy, which can mask or sometimes mimic inflammation. Quantitative MRI (qMRI) methods provide a means to better differentiate mural inflammation from fibrosis and improve quantification of these processes. qMRI may also provide more objective measures of disease activity and enable better tailoring of treatment. Here, we review quantitative MRI methods for imaging the small bowel in CD and consider the path to their clinical translation. LEVEL OF EVIDENCE: 2 TECHNICAL EFFICACY: Stage 2.

Cross-vendor multiparametric mapping of the human brain using 3D-QALAS: A multicenter and multivendor study

PURPOSE

To evaluate a vendor-agnostic multiparametric mapping scheme based on 3D quantification using an interleaved Look-Locker acquisition sequence with a T2 preparation pulse (3D-QALAS) for whole-brain T1, T2, and proton density (PD) mapping.

METHODS

This prospective, multi-institutional study was conducted between September 2021 and February 2022 using five different 3T systems from four prominent MRI vendors. The accuracy of this technique was evaluated using a standardized MRI system phantom. Intra-scanner repeatability and inter-vendor reproducibility of T1, T2, and PD values were evaluated in 10 healthy volunteers (6 men; mean age ± SD, 28.0 ± 5.6 y) who underwent scan-rescan sessions on each scanner (total scans = 100). To evaluate the feasibility of 3D-QALAS, nine patients with multiple sclerosis (nine women; mean age ± SD, 48.2 ± 11.5 y) underwent imaging examination on two 3T MRI systems from different manufacturers.

RESULTS

Quantitative maps obtained with 3D-QALAS showed high linearity (R2  = 0.998 and 0.998 for T1 and T2, respectively) with respect to reference measurements. The mean intra-scanner coefficients of variation for each scanner and structure ranged from 0.4% to 2.6%. The mean structure-wise test-retest repeatabilities were 1.6%, 1.1%, and 0.7% for T1, T2, and PD, respectively. Overall, high inter-vendor reproducibility was observed for all parameter maps and all structure measurements, including white matter lesions in patients with multiple sclerosis.

CONCLUSION

The vendor-agnostic multiparametric mapping technique 3D-QALAS provided reproducible measurements of T1, T2, and PD for human tissues within a typical physiological range using 3T scanners from four different MRI manufacturers.

Breast density quantification in dual-energy mammography using virtual anthropomorphic phantoms

PURPOSE

Breast density is a significant risk factor for breast cancer and can impact the sensitivity of screening mammography. Area-based breast density measurements may not provide an accurate representation of the tissue distribution, therefore volumetric breast density (VBD) measurements are preferred. Dual-energy mammography enables volumetric measurements without additional assumptions about breast shape. In this work we evaluated the performance of a dual-energy decomposition technique for determining VBD by applying it to virtual anthropomorphic phantoms.

METHODS

The dual-energy decomposition formalism was used to quantify VBD on simulated dual-energy images of anthropomorphic virtual phantoms with known tissue distributions. We simulated 150 phantoms with volumes ranging from 50 to 709 mL and VBD ranging from 15% to 60%. Using these results, we validated a correction for the presence of skin and assessed the method's intrinsic bias and variability. As a proof of concept, the method was applied to 14 sets of clinical dual-energy images, and the resulting breast densities were compared to magnetic resonance imaging (MRI) measurements.

RESULTS

Virtual phantom VBD measurements exhibited a strong correlation (Pearson'sr > 0.95 $r > 0.95$ ) with nominal values. The proposed skin correction eliminated the variability due to breast size and reduced the bias in VBD to a constant value of -2%. Disagreement between clinical VBD measurements using MRI and dual-energy mammography was under 10%, and the difference in the distributions was statistically non-significant. VBD measurements in both modalities had a moderate correlation (Spearman'sρ $\rho \ $ = 0.68).

CONCLUSIONS

Our results in virtual phantoms indicate that the material decomposition method can produce accurate VBD measurements if the presence of a third material (skin) is considered. The results from our proof of concept showed agreement between MRI and dual-energy mammography VBD. Assessment of VBD using dual-energy images could provide complementary information in dual-energy mammography and tomosynthesis examinations.

Inter- and Intra-Patient Repeatability of Radiomic Features from Multiparametric Whole-Body MRI in Patients with Metastatic Prostate Cancer

(1) Background: We assessed the test-re-test repeatability of radiomics in metastatic castration-resistant prostate cancer (mCPRC) bone disease on whole-body diffusion-weighted (DWI) and T1-weighted Dixon MRI. (2) Methods: In 10 mCRPC patients, 1.5 T MRI, including DWI and T1-weighted gradient-echo Dixon sequences, was performed twice on the same day. Apparent diffusion coefficient (ADC) and relative fat-fraction-percentage (rFF%) maps were calculated. Per study, up to 10 target bone metastases were manually delineated on DWI and Dixon images. All 106 radiomic features included in the Pyradiomics toolbox were derived for each target volume from the ADC and rFF% maps. To account for inter- and intra-patient measurement repeatability, the log-transformed individual target measurements were fitted to a hierarchical model, represented as a Bayesian network. Repeatability measurements, including the intraclass correlation coefficient (ICC), were derived. Feature ICCs were compared with mean ADC and rFF ICCs. (3) Results: A total of 65 DWI and 47 rFF% targets were analysed. There was no significant bias for any features. Pairwise correlation revealed fifteen ADC and fourteen rFF% feature sub-groups, without specific patterns between feature classes. The median intra-patient ICC was generally higher than the inter-patient ICC. Features that describe extremes in voxel values (minimum, maximum, range, skewness, and kurtosis) showed generally lower ICCs. Several mostly shape-based texture features were identified, which showed high inter- and intra-patient ICCs when compared with the mean ADC or mean rFF%, respectively. (4) Conclusions: Pyradiomics texture features of mCRPC bone metastases varied greatly in inter- and intra-patient repeatability. Several features demonstrated good repeatability, allowing for further exploration as diagnostic parameters in mCRPC bone disease.

Hardware and Software Setup for Quantitative 23Na Magnetic Resonance Imaging at 3T: A Phantom Study

Magnetic resonance (MR) with sodium (23Na) is a noninvasive tool providing quantitative biochemical information regarding physiology, cellular metabolism, and viability, with the potential to extend MR beyond anatomical proton imaging. However, when using clinical scanners, the low detectable 23Na signal and the low 23Na gyromagnetic ratio require the design of dedicated radiofrequency (RF) coils tuned to the 23Na Larmor frequency and sequences, as well as the development of dedicated phantoms for testing the image quality, and an MR scanner with multinuclear spectroscopy (MNS) capabilities. In this work, we propose a hardware and software setup for evaluating the potential of 23Na magnetic resonance imaging (MRI) with a clinical scanner. In particular, the reliability of the proposed setup and the reproducibility of the measurements were verified by multiple acquisitions from a 3T MR scanner using a homebuilt RF volume coil and a dedicated sequence for the imaging of a phantom specifically designed for evaluating the accuracy of the technique. The final goal of this study is to propose a setup for standardizing clinical and research 23Na MRI protocols.

Deep learning-based segmentation of whole-body fetal MRI and fetal weight estimation: assessing performance, repeatability, and reproducibility

OBJECTIVES

To develop a deep-learning method for whole-body fetal segmentation based on MRI; to assess the method's repeatability, reproducibility, and accuracy; to create an MRI-based normal fetal weight growth chart; and to assess the sensitivity to detect fetuses with growth restriction (FGR).

METHODS

Retrospective data of 348 fetuses with gestational age (GA) of 19-39 weeks were included: 249 normal appropriate for GA (AGA), 19 FGR, and 80 Other (having various imaging abnormalities). A fetal whole-body segmentation model with a quality estimation module was developed and evaluated in 169 cases. The method was evaluated for its repeatability (repeated scans within the same scanner, n = 22), reproducibility (different scanners, n = 6), and accuracy (compared with birth weight, n = 7). A normal MRI-based growth chart was derived.

RESULTS

The method achieved a Dice = 0.973, absolute volume difference ratio (VDR) = 1.8% and VDR mean difference = 0.75% ([Formula: see text]: - 3.95%, 5.46), and high agreement with the gold standard. The method achieved a repeatability coefficient = 4.01%, ICC = 0.99, high reproducibility with a mean difference = 2.21% ([Formula: see text]: - 1.92%, 6.35%), and high accuracy with a mean difference between estimated fetal weight (EFW) and birth weight of - 0.39% ([Formula: see text]: - 8.23%, 7.45%). A normal growth chart (n = 246) was consistent with four ultrasound charts. EFW based on MRI correctly predicted birth-weight percentiles for all 18 fetuses ≤ 10thpercentile and for 14 out of 17 FGR fetuses below the 3rd percentile. Six fetuses referred to MRI as AGA were found to be < 3rd percentile.

CONCLUSIONS

The proposed method for automatic MRI-based EFW demonstrated high performance and sensitivity to identify FGR fetuses.

CLINICAL RELEVANCE STATEMENT

Results from this study support the use of the automatic fetal weight estimation method based on MRI for the assessment of fetal development and to detect fetuses at risk for growth restriction.

KEY POINTS

• An AI-based segmentation method with a quality assessment module for fetal weight estimation based on MRI was developed, achieving high repeatability, reproducibility, and accuracy. • An MRI-based fetal weight growth chart constructed from a large cohort of normal and appropriate gestational-age fetuses is proposed. • The method showed a high sensitivity for the diagnosis of small fetuses suspected of growth restriction.

Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data

Reliable performance of PET segmentation algorithms on clinically relevant tasks is required for their clinical translation. However, these algorithms are typically evaluated using figures of merit (FoMs) that are not explicitly designed to correlate with clinical task performance. Such FoMs include the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the Hausdorff distance (HD). The objective of this study was to investigate whether evaluating PET segmentation algorithms using these task-agnostic FoMs yields interpretations consistent with evaluation on clinically relevant quantitative tasks. Methods: We conducted a retrospective study to assess the concordance in the evaluation of segmentation algorithms using the DSC, JSC, and HD and on the tasks of estimating the metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumors from PET images of patients with non-small cell lung cancer. The PET images were collected from the American College of Radiology Imaging Network 6668/Radiation Therapy Oncology Group 0235 multicenter clinical trial data. The study was conducted in 2 contexts: (1) evaluating conventional segmentation algorithms, namely those based on thresholding (SUVmax40% and SUVmax50%), boundary detection (Snakes), and stochastic modeling (Markov random field-Gaussian mixture model); (2) evaluating the impact of network depth and loss function on the performance of a state-of-the-art U-net-based segmentation algorithm. Results: Evaluation of conventional segmentation algorithms based on the DSC, JSC, and HD showed that SUVmax40% significantly outperformed SUVmax50%. However, SUVmax40% yielded lower accuracy on the tasks of estimating MTV and TLG, with a 51% and 54% increase, respectively, in the ensemble normalized bias. Similarly, the Markov random field-Gaussian mixture model significantly outperformed Snakes on the basis of the task-agnostic FoMs but yielded a 24% increased bias in estimated MTV. For the U-net-based algorithm, our evaluation showed that although the network depth did not significantly alter the DSC, JSC, and HD values, a deeper network yielded substantially higher accuracy in the estimated MTV and TLG, with a decreased bias of 91% and 87%, respectively. Additionally, whereas there was no significant difference in the DSC, JSC, and HD values for different loss functions, up to a 73% and 58% difference in the bias of the estimated MTV and TLG, respectively, existed. Conclusion: Evaluation of PET segmentation algorithms using task-agnostic FoMs could yield findings discordant with evaluation on clinically relevant quantitative tasks. This study emphasizes the need for objective task-based evaluation of image segmentation algorithms for quantitative PET.

Feasibility of dynamic T2 *-based oxygen-enhanced lung MRI at 3T

PURPOSE

To demonstrate proof-of-concept of a T2 *-sensitized oxygen-enhanced MRI (OE-MRI) method at 3T by assessing signal characteristics, repeatability, and reproducibility of dynamic lung OE-MRI metrics in healthy volunteers.

METHODS

We performed sequence-specific simulations for protocol optimisation and acquired free-breathing OE-MRI data from 16 healthy subjects using a dual-echo RF-spoiled gradient echo approach at 3T across two institutions. Non-linear registration and tissue density correction were applied. Derived metrics included percent signal enhancement (PSE), ∆R2 * and wash-in time normalized for breathing rate (τ-nBR). Inter-scanner reproducibility and intra-scanner repeatability were evaluated using intra-class correlation coefficient (ICC), repeatability coefficient, reproducibility coefficient, and Bland-Altman analysis.

RESULTS

Simulations and experimental data show negative contrast upon oxygen inhalation, due to substantial dominance of ∆R2 * at TE > 0.2 ms. Density correction improved signal fluctuations. Density-corrected mean PSE values, aligned with simulations, display TE-dependence, and an anterior-to-posterior PSE reduction trend at TE1 . ∆R2 * maps exhibit spatial heterogeneity in oxygen delivery, featuring anterior-to-posterior R2 * increase. Mean T2 * values across 32 scans were 0.68 and 0.62 ms for pre- and post-O2 inhalation, respectively. Excellent or good agreement emerged from all intra-, inter-scanner and inter-rater variability tests for PSE and ∆R2 *. However, ICC values for τ-nBR demonstrated limited agreement between repeated measures.

CONCLUSION

Our results demonstrate the feasibility of a T2 *-weighted method utilizing a dual-echo RF-spoiled gradient echo approach, simultaneously capturing PSE, ∆R2 * changes, and oxygen wash-in during free-breathing. The excellent or good repeatability and reproducibility on intra- and inter-scanner PSE and ∆R2 * suggest potential utility in multi-center clinical applications.

Independent component analysis (ICA) applied to dynamic oxygen-enhanced MRI (OE-MRI) for robust functional lung imaging at 3 T

PURPOSE

Dynamic lung oxygen-enhanced MRI (OE-MRI) is challenging due to the presence of confounding signals and poor signal-to-noise ratio, particularly at 3 T. We have created a robust pipeline utilizing independent component analysis (ICA) to automatically extract the oxygen-induced signal change from confounding factors to improve the accuracy and sensitivity of lung OE-MRI.

METHODS

Dynamic OE-MRI was performed on healthy participants using a dual-echo multi-slice spoiled gradient echo sequence at 3 T and cyclical gas delivery. ICA was applied to each echo within a thoracic mask. The ICA component relating to the oxygen-enhancement signal was automatically identified using correlation analysis. The oxygen-enhancement component was reconstructed, and the percentage signal enhancement (PSE) was calculated. The lung PSE of current smokers was compared with nonsmokers; scan-rescan repeatability, ICA pipeline repeatability, and reproducibility between two vendors were assessed.

RESULTS

ICA successfully extracted a consistent oxygen-enhancement component for all participants. Lung tissue and oxygenated blood displayed the opposite oxygen-induced signal enhancements. A significant difference in PSE was observed between the lungs of current smokers and nonsmokers. The scan-rescan repeatability and the ICA pipeline repeatability were good.

CONCLUSION

The developed pipeline demonstrated sensitivity to the signal enhancements of the lung tissue and oxygenated blood at 3 T. The difference in lung PSE between current smokers and nonsmokers indicates a likely sensitivity to lung function alterations that may be seen in mild pathology, supporting future use of our methods in patient studies.

Understanding ADC variation by fat content effect using a dual-function MRI phantom

BACKGROUND

A dual-function phantom designed to quantify the apparent diffusion coefficient (ADC) in different fat contents (FCs) and glass bead densities (GBDs) to simulate the human tissues has not been documented yet. We propose a dual-function phantom to quantify the FC and to measure the ADC at different FCs and different GBDs.

METHODS

A fat-containing diffusion phantom comprised by 30 glass-bead-containing fat-water emulsions consisting of six different FCs (0, 10, 20, 30, 40, and 50%) multiplied by five different GBDs (0, 0.1, 0.25, 0.5, and 1.0 g/50 mL). The FC and ADC were measured by the "iterative decomposition of water and fat with echo asymmetry and least squares estimation-IQ," IDEAL-IQ, and single-shot echo-planar diffusion-weighted imaging, SS-EP-DWI, sequences, respectively. Linear regression analysis was used to evaluate the relationship among the fat fraction (FF) measured by IDEAL-IQ, GBD, and ADC.

RESULTS

The ADC was significantly, negatively, and linearly associated with the FF (the linear slope ranged from -0.005 to -0.017, R2 = 0.925 to 0.986, all p < 0.001). The slope of the linear relationship between the ADC and the FF, however, varied among different GBDs (the higher the GBD, the lower the slope). ADCs among emulsions across different GBDs and FFs were overlapped. Emulsions with low GBDs plus high FFs shared a same lower ADC range with those with median or high GBDs plus median or lower FFs.

CONCLUSIONS

A novel dual-function phantom simulating the human tissues allowed to quantify the influence of FC and GBD on ADC.

RELEVANCE STATEMENT

The study developed an innovative dual-function MRI phantom to explore the impact of FC on ADC variation that can affect clinical results. The results revealed the superimposed effect on FF and GBD density on ADC measurements.

KEY POINTS

• A dual-function phantom made of glass bead density (GBD) and fat fraction (FF) emulsion has been developed. • Apparent diffusion coefficient (ADC) values are determined by GBD and FF. • The dual-function phantom showed the mutual ADC addition between FF and GBD.

Simulation of ultrasonic scattering from scatterer size distributions using Field II

Quantitative analysis of radio frequency (RF) signals obtained from ultrasound scanners can yield objective parameters that are gaining clinical relevance as imaging biomarkers. These include the backscatter coefficient (BSC) and the effective scatterer diameter (ESD). Biomarker validation is typically performed in phantoms which do not provide the flexibility of systematic variation of scattering properties. Computer simulations, such as those from the ultrasound simulator Field II, can allow more flexibility. However, Field II does not allow simulation of RF data from a distribution of scatterers with finite size. In this work, a simulation method is presented which builds upon previous work by including Faran theory models representative of distributions of scatterer size. These are systematically applied to RF data simulated in Field II. The method is validated by measuring the root mean square error of the estimated BSC and percent bias of the ESD and comparing to experimental results. The results indicate the method accurately simulates distributions of scatterer sizes and provides scattering similar to that seen in data from clinical scanners. Because Field II is widely used by the ultrasound community, this method can be adopted to aid in validation of quantitative ultrasound imaging biomarkers.

Assessment of Feasibility and Interscan Variability of Short-time Cardiac MRI for Cardiotoxicity Evaluation in Breast Cancer

Purpose To investigate the feasibility and interscan variability of short-time cardiac MRI protocol after chemotherapy in individuals with breast cancer. Materials and Methods A total of 13 healthy female controls (mean age, 52.4 years ± 13.2 [SD]) and 85 female participants with breast cancer (mean age, 51.8 years ± 9.9) undergoing chemotherapy prospectively underwent routine breast MRI and short-time cardiac MRI using a 3-T scanner with peripheral pulse gating in the prone position. Interscan, intercoil, and interobserver reproducibility and variability of native T1 and extracellular volume (ECV), as well as ventricular functional parameters, were measured using the intraclass correlation coefficient (ICC), standard error of measurement (SEM), or coefficient of variation (CoV). Results Left ventricular functional parameters had excellent interscan reproducibility (ICC ≥ 0.80). Left ventricular ejection fraction showed low interscan variability in control and chemotherapy participants (SEM, 2.0 and 1.2; CoV, 3.1 and 1.9, respectively). Native T1 showed excellent interscan (ICC, 0.75) and intercoil (ICC, 0.81) reproducibility in the control group and good interscan reproducibility (ICC, 0.72 and 0.73, respectively) in the participants undergoing immediate and remote chemotherapy. Interscan reproducibility for ECV was excellent in the control group and in the remote chemotherapy group (ICC, 0.93 and 0.88, respectively) and fair in the immediate chemotherapy group (ICC, 0.52). In the regional analysis, interscan repeatability and variability of native T1 and ECV were superior in the anteroseptum or inferoseptum than in other segments in the immediate chemotherapy group. Native T1 and ECV had good to excellent interobserver agreement across all groups. Conclusion Short-time cardiac MRI showed excellent results for interscan, intercoil, and interobserver reproducibility and variability for ventricular functional or tissue characterization parameters, suggesting that this modality is feasible for routine surveillance of cardiotoxicity evaluation in individuals with breast cancer. Keywords: Cardiac MRI, Heart, Cardiomyopathy ClinicalTrials.gov registration no. NCT03301389 Supplemental material is available for this article. © RSNA, 2024.

A statistical framework for planning and analysing test-retest studies of repeatability

There is an increasing number of potential quantitative biomarkers that could allow for early assessment of treatment response or disease progression. However, measurements of such biomarkers are subject to random variability. Hence, differences of a biomarker in longitudinal measurements do not necessarily represent real change but might be caused by this random measurement variability. Before utilizing a quantitative biomarker in longitudinal studies, it is therefore essential to assess the measurement repeatability. Measurement repeatability obtained from test-retest studies can be quantified by the repeatability coefficient, which is then used in the subsequent longitudinal study to determine if a measured difference represents real change or is within the range of expected random measurement variability. The quality of the point estimate of the repeatability coefficient, therefore, directly governs the assessment quality of the longitudinal study. Repeatability coefficient estimation accuracy depends on the case number in the test-retest study, but despite its pivotal role, no comprehensive framework for sample size calculation of test-retest studies exists. To address this issue, we have established such a framework, which allows for flexible sample size calculation of test-retest studies, based upon newly introduced criteria concerning assessment quality in the longitudinal study. This also permits retrospective assessment of prior test-retest studies.

Palm-Sized Wireless Transient Elastography System with Real-Time B-Mode Ultrasound Imaging Guidance: Toward Point-of-Care Liver Fibrosis Assessment

Transient elastography (TE), recommended by the WHO, is an established method for characterizing liver fibrosis via liver stiffness measurement (LSM). However, technical barriers remain towards point-of-care application, as conventional TE requires wired connections, possesses a bulky size, and lacks adequate imaging guidance for precise liver localization. In this work, we report the design, phantom validation, and clinical evaluation of a palm-sized TE system that enables simultaneous B-mode imaging and LSM. The performance of this system was validated experimentally using tissue-equivalent reference phantoms (1.45-75 kPa). Comparative studies against other liver elastography techniques, including conventional TE and two-dimensional shear wave elastography (2D-SWE), were performed to evaluate its reliability and validity in adults with various chronic liver diseases. Intra- and inter-operator reliability of LSM were established by an elastography expert and a novice. A good agreement was observed between the Young's modulus reported by the phantom manufacturer and this system (bias: 1.1-8.6%). Among 121 patients, liver stiffness measured by this system and conventional TE were highly correlated (r = 0.975) and strongly agreed with each other (mean difference: -0.77 kPa). Inter-correlation of this system with conventional TE and 2D-SWE was observed. Excellent-to-good operator reliability was demonstrated in 60 patients (ICCs: 0.824-0.913). We demonstrated the feasibility of employing a fully integrated phased array probe for reliable and valid LSM, guided by real-time B-mode imaging of liver anatomy. This system represents the first technical advancement toward point-of-care liver fibrosis assessment. Its small footprint, along with B-mode guidance capability, improves examination efficiency and scales up screening for liver fibrosis.

Functional information guided adaptive radiation therapy

Introduction

Functional informaton is introduced as the mechanism to adapt cancer therapies uniquely to individual patients based on changes defined by qualified tumor biomarkers.

Methods

To demonstrate the methodology, a tumor volume biomarker model, characterized by a tumor volume reduction rate coefficient, is used to adapt a tumor cell survival bioresponse radiotherapy model in terms of therapeutic radiation dose. Tumor volume, acquired from imaging data, serves as a surrogate measurement for tumor cell death, but the biomarker model derived from this data cannot be used to calculate the radiation dose absorbed by the target tumor. However, functional information does provide a mathematical connection between the tumor volume biomarker model and the tumor cell survival bioresponse model by quantifying both data sets in the units of information, thus creating an analytic conduit from bioresponse to biomarker.

Results

The information guided process for individualized dose adaptations using information values acquired from the tumor cell survival bioresponse model and the tumor volume biomarker model are presented in detailed form by flowchart and tabular data. Clinical data are used to generate a presentation that assists investigator application of the information guided methodology to adaptive cancer therapy research.

Conclusions

Information guided adaptation of bioresponse using surrogate data is extensible across multiple research fields because functional information mathematically connects disparate bioresponse and biomarker data sets. Thus, functional information offers adaptive cancer therapy by mathematically connecting immunotherapy, chemotherapy, and radiotherapy cancer treatment processes to implement individualized treatment plans.

Efficiency and Accuracy Evaluation of Multiple Diffusion-Weighted MRI Techniques Across Different Scanners

BACKGROUND

The choice between different diffusion-weighted imaging (DWI) techniques is difficult as each comes with tradeoffs for efficient clinical routine imaging and apparent diffusion coefficient (ADC) accuracy.

PURPOSE

To quantify signal-to-noise-ratio (SNR) efficiency, ADC accuracy, artifacts, and distortions for different DWI acquisition techniques, coils, and scanners.

STUDY TYPE

Phantom, in vivo intraindividual biomarker accuracy between DWI techniques and independent ratings.

POPULATION/PHANTOMS

NIST diffusion phantom. 51 Patients: 40 with prostate cancer and 11 with head-and-neck cancer at 1.5 T FIELD STRENGTH/SEQUENCE: Echo planar imaging (EPI): 1.5 T and 3 T Siemens; 3 T Philips. Distortion-reducing: RESOLVE (1.5 and 3 T Siemens); Turbo Spin Echo (TSE)-SPLICE (3 T Philips). Small field-of-view (FOV): ZoomitPro (1.5 T Siemens); IRIS (3 T Philips). Head-and-neck and flexible coils.

ASSESSMENT

SNR Efficiency, geometrical distortions, and susceptibility artifacts were quantified for different b-values in a phantom. ADC accuracy/agreement was quantified in phantom and for 51 patients. In vivo image quality was independently rated by four experts.

STATISTICAL TESTS

QIBA methodology for accuracy: trueness, repeatability, reproducibility, Bland-Altman 95% Limits-of-Agreement (LOA) for ADC. Wilcoxon Signed-Rank and student tests on P < 0.05 level.

RESULTS

The ZoomitPro small FOV sequence improved b-image efficiency by 8%-14%, reduced artifacts and observer scoring for most raters at the cost of smaller FOV compared to EPI. The TSE-SPLICE technique reduced artifacts almost completely at a 24% efficiency cost compared to EPI for b-values ≤500 sec/mm2 . Phantom ADC 95% LOA trueness were within ±0.03 × 10-3  mm2 /sec except for small FOV IRIS. The in vivo ADC agreement between techniques, however, resulted in 95% LOAs in the order of ±0.3 × 10-3  mm2 /sec with up to 0.2 × 10-3  mm2 /sec of bias.

DATA CONCLUSION

ZoomitPro for Siemens and TSE SPLICE for Philips resulted in a trade-off between efficiency and artifacts. Phantom ADC quality control largely underestimated in vivo accuracy: significant ADC bias and variability was found between techniques in vivo.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY STAGE: 2.

Accuracy and Precision of Iodine Quantification in Subtracted Micro-Computed Tomography: Effect of Reconstruction and Noise Removal Algorithms

PURPOSE

To evaluate the effect of reconstruction and noise removal algorithms on the accuracy and precision of iodine concentration (CI) quantified with subtracted micro-computed tomography (micro-CT).

PROCEDURES

Two reconstruction algorithms were evaluated: a filtered backprojection (FBP) algorithm and a simultaneous iterative reconstruction technique (SIRT) algorithm. A 3D bilateral filter (BF) was used for noise removal. A phantom study evaluated and compared the image quality, and the accuracy and precision of CI in four scenarios: filtered FBP, filtered SIRT, non-filtered FBP, and non-filtered SIRT. In vivo experiments were performed in an animal model of chemically-induced mammary cancer.

RESULTS

Linear relationships between the measured and nominal CI values were found for all the scenarios in the phantom study (R2 > 0.95). SIRT significantly improved the accuracy and precision of CI compared to FBP, as given by their lower bias (adj. p-value = 0.0308) and repeatability coefficient (adj. p-value < 0.0001). Noise removal enabled a significant decrease in bias in filtered SIRT images only; non-significant differences were found for the repeatability coefficient. The phantom and in vivo studies showed that CI is a reproducible imaging parameter for all the scenarios (Pearson r > 0.99, p-value < 0.001). The contrast-to-noise ratio showed non-significant differences among the evaluated scenarios in the phantom study, while a significant improvement was found in the in vivo study when SIRT and BF algorithms were used.

CONCLUSIONS

SIRT and BF algorithms improved the accuracy and precision of CI compared to FBP and non-filtered images, which encourages their use in subtracted micro-CT imaging.

Direct comparison and reproducibility of two segmentation methods for multicompartment dosimetry: round robin study on radioembolization treatment planning in hepatocellular carcinoma

PURPOSE

Investigate reproducibility of two segmentation methods for multicompartment dosimetry, including normal tissue absorbed dose (NTAD) and tumour absorbed dose (TAD), in hepatocellular carcinoma patients treated with yttrium-90 (90Y) glass microspheres.

METHODS

TARGET was a retrospective investigation in 209 patients with < 10 tumours per lobe and at least one tumour ≥ 3 cm ± portal vein thrombosis. Dosimetry was compared using two distinct segmentation methods: anatomic (CT/MRI-based) and count threshold-based on pre-procedural 99mTc-MAA SPECT. In a round robin substudy in 20 patients with ≤ 5 unilobar tumours, the inter-observer reproducibility of eight reviewers was evaluated by computing reproducibility coefficient (RDC) of volume and absorbed dose for whole liver, whole liver normal tissue, perfused normal tissue, perfused liver, total perfused tumour, and target lesion. Intra-observer reproducibility was based on second assessments in 10 patients ≥ 2 weeks later.

RESULTS

99mTc-MAA segmentation calculated higher absorbed doses compared to anatomic segmentation (n = 209), 43.9% higher for TAD (95% limits of agreement [LoA]: - 49.0%, 306.2%) and 21.3% for NTAD (95% LoA: - 67.6%, 354.0%). For the round robin substudy (n = 20), inter-observer reproducibility was better for anatomic (RDC range: 1.17 to 3.53) than 99mTc-MAA SPECT segmentation (1.29 to 7.00) and similar between anatomic imaging modalities (CT: 1.09 to 3.56; MRI: 1.24 to 3.50). Inter-observer reproducibility was better for larger volumes. Perfused normal tissue volume RDC was 1.95 by anatomic and 3.19 by 99mTc-MAA SPECT, with corresponding absorbed dose RDC 1.46 and 1.75. Total perfused tumour volume RDC was higher, 2.92 for anatomic and 7.0 by 99mTc-MAA SPECT with corresponding absorbed dose RDC of 1.84 and 2.78. Intra-observer variability was lower for perfused NTAD (range: 14.3 to 19.7 Gy) than total perfused TAD (range: 42.8 to 121.4 Gy).

CONCLUSION

Anatomic segmentation-based dosimetry, versus 99mTc-MAA segmentation, results in lower absorbed doses with superior reproducibility. Higher volume compartments, such as normal tissue versus tumour, exhibit improved reproducibility.

TRIAL REGISTRATION

NCT03295006.

Inter- and Intra-Operator Variability of Regularized Backscatter Quantitative Ultrasound for the Characterization of Breast Masses

OBJECTIVES

Here we report on the intra- and inter-operator variability of the backscatter coefficient (BSC) estimated with a new low-variance quantitative ultrasound (QUS) approach applied to breast lesions in vivo.

METHODS

Radiofrequency (RF) echo signals were acquired from 29 BIRADS 4 and 5 breast lesions in 2 sequential cohorts following 2 imaging protocols: cohort 1) radial and antiradial views, and cohort 2) short- and long-axis views. Protocol 2 was implemented after retraining and discussion on how to improve reproducibility. Each patient was scanned by at least 2 of 3 radiologists; each performed 3 acquisitions with transducer and patient repositioning in between acquisitions. BSC was estimated using a low-variance QUS approach based on regularization. Intra- and inter-operator variability of the intra-lesion median BSC was evaluated with a multifactorial ANOVA test (P-values) and the intraclass correlation coefficient (ICC).

RESULTS

Inter-operator variability was only significant in the first protocol (P < .007); ICCinter  = .77 (95% CI .71-.82), indicating good inter-operator agreement. In the second protocol, the inter-operator variability was not significant (P > .05) and agreement was excellent (ICCinter  = .92 [.89-.94]). In both protocols, the intra-operator variability was not significant.

CONCLUSIONS

Our findings demonstrate the need for standardizing image acquisition protocols for backscatter-based QUS to reduce inter-operator variability and ensure its successful translation to the characterization of suspicious breast masses.

Quantitative diffusion-weighted MRI response assessment in rhabdomyosarcoma: an international retrospective study on behalf of the European paediatric Soft tissue sarcoma Study Group Imaging Committee

OBJECTIVE

To investigate the feasibility of diffusion-weighted magnetic resonance imaging (DW-MRI) as a predictive imaging marker after neoadjuvant chemotherapy in patients with rhabdomyosarcoma.

MATERIAL AND METHODS

We performed a multicenter retrospective study including pediatric, adolescent and young adult patients with rhabdomyosarcoma, Intergroup Rhabdomyosarcoma Study group III/IV, treated according to the European paediatric Soft tissue sarcoma Study Group (EpSSG) RMS2005 or MTS2008 studies. DW-MRI was performed according to institutional protocols. We performed two-dimensional single-slice tumor delineation. Areas of necrosis or hemorrhage were delineated to be excluded in the primary analysis. Mean, median and 5th and 95th apparent diffusion coefficient (ADC) were extracted.

RESULTS

Of 134 included patients, 82 had measurable tumor at diagnosis and response and DW-MRI scans of adequate quality and were included in the analysis. Technical heterogeneity in scan acquisition protocols and scanners was observed. Mean ADC at diagnosis was 1.1 (95% confidence interval [CI]: 1.1-1.2) (all ADC expressed in * 10-3 mm2/s), versus 1.6 (1.5-1.6) at response assessment. The 5th percentile ADC was 0.8 (0.7-0.9) at diagnosis and 1.1 (1.0-1.2) at response. Absolute change in mean ADC after neoadjuvant chemotherapy was 0.4 (0.3-0.5). Exploratory analyses for association between ADC and clinical parameters showed a significant difference in mean ADC at diagnosis for alveolar versus embryonal histology. Landmark analysis at nine weeks after the date of diagnosis showed no significant association (hazard ratio 1.3 [0.6-3.2]) between the mean ADC change and event-free survival.

CONCLUSION

A significant change in the 5th percentile and the mean ADC after chemotherapy was observed. Strong heterogeneity was identified in DW-MRI acquisition protocols between centers and in individual patients.

Repeatability of MRI Biomarkers in Nonalcoholic Fatty Liver Disease: The NIMBLE Consortium

Background There is a need for reliable noninvasive methods for diagnosing and monitoring nonalcoholic fatty liver disease (NAFLD). Thus, the multidisciplinary Non-invasive Biomarkers of Metabolic Liver disease (NIMBLE) consortium was formed to identify and advance the regulatory qualification of NAFLD imaging biomarkers. Purpose To determine the different-day same-scanner repeatability coefficient of liver MRI biomarkers in patients with NAFLD at risk for steatohepatitis. Materials and Methods NIMBLE 1.2 is a prospective, observational, single-center short-term cross-sectional study (October 2021 to June 2022) in adults with NAFLD across a spectrum of low, intermediate, and high likelihood of advanced fibrosis as determined according to the fibrosis based on four factors (FIB-4) index. Participants underwent up to seven MRI examinations across two visits less than or equal to 7 days apart. Standardized imaging protocols were implemented with six MRI scanners from three vendors at both 1.5 T and 3 T, with central analysis of the data performed by an independent reading center (University of California, San Diego). Trained analysts, who were blinded to clinical data, measured the MRI proton density fat fraction (PDFF), liver stiffness at MR elastography (MRE), and visceral adipose tissue (VAT) for each participant. Point estimates and CIs were calculated using χ2 distribution and statistical modeling for pooled repeatability measures. Results A total of 17 participants (mean age, 58 years ± 8.5 [SD]; 10 female) were included, of which seven (41.2%), six (35.3%), and four (23.5%) participants had a low, intermediate, or high likelihood of advanced fibrosis, respectively. The different-day same-scanner mean measurements were 13%-14% for PDFF, 6.6 L for VAT, and 3.15 kPa for two-dimensional MRE stiffness. The different-day same-scanner repeatability coefficients were 0.22 L (95% CI: 0.17, 0.29) for VAT, 0.75 kPa (95% CI: 0.6, 0.99) for MRE stiffness, 1.19% (95% CI: 0.96, 1.61) for MRI PDFF using magnitude reconstruction, 1.56% (95% CI: 1.26, 2.07) for MRI PDFF using complex reconstruction, and 19.7% (95% CI: 15.8, 26.2) for three-dimensional MRE shear modulus. Conclusion This preliminary study suggests that thresholds of 1.2%-1.6%, 0.22 L, and 0.75 kPa for MRI PDFF, VAT, and MRE, respectively, should be used to discern measurement error from real change in patients with NAFLD. ClinicalTrials.gov registration no. NCT05081427 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Kozaka and Matsui in this issue.

Blood-brain barrier water exchange measurements using FEXI: Impact of modeling paradigm and relaxation time effects

PURPOSE

To evaluate potential modeling paradigms and the impact of relaxation time effects on human blood-brain barrier (BBB) water exchange measurements using FEXI (BBB-FEXI), and to quantify the accuracy, precision, and repeatability of BBB-FEXI exchange rate estimates at 3 T $$ \mathrm{T} $$ .

METHODS

Three modeling paradigms were evaluated: (i) the apparent exchange rate (AXR) model; (ii) a two-compartment model (2 CM $$ 2\mathrm{CM} $$ ) explicitly representing intra- and extravascular signal components, and (iii) a two-compartment model additionally accounting for finite compartmentalT 1 $$ {\mathrm{T}}_1 $$ andT 2 $$ {\mathrm{T}}_2 $$ relaxation times (2 CM r $$ 2{\mathrm{CM}}_r $$ ). Each model had three free parameters. Simulations quantified biases introduced by the assumption of infinite relaxation times in the AXR and2 CM $$ 2\mathrm{CM} $$ models, as well as the accuracy and precision of all three models. The scan-rescan repeatability of all paradigms was quantified for the first time in vivo in 10 healthy volunteers (age range 23-52 years; five female).

RESULTS

The assumption of infinite relaxation times yielded exchange rate errors in simulations up to 42%/14% in the AXR/2 CM $$ 2\mathrm{CM} $$ models, respectively. Accuracy was highest in the compartmental models; precision was best in the AXR model. Scan-rescan repeatability in vivo was good for all models, with negligible bias and repeatability coefficients in grey matter ofRC AXR = 0 . 43 $$ {\mathrm{RC}}_{\mathrm{AXR}}=0.43 $$  s - 1 $$ {\mathrm{s}}^{-1} $$ ,RC 2 CM = 0 . 51 $$ {\mathrm{RC}}_{2\mathrm{CM}}=0.51 $$  s - 1 $$ {\mathrm{s}}^{-1} $$ , andRC 2 CM r = 0 . 61 $$ {\mathrm{RC}}_{2{\mathrm{CM}}_r}=0.61 $$  s - 1 $$ {\mathrm{s}}^{-1} $$ .

CONCLUSION

Compartmental modelling of BBB-FEXI signals can provide accurate and repeatable measurements of BBB water exchange; however, relaxation time and partial volume effects may cause model-dependent biases.

First-in-human technique translation of oxygen-enhanced MRI to an MR Linac system in patients with head and neck cancer

BACKGROUND AND PURPOSE

Tumour hypoxia is prognostic in head and neck cancer (HNC), associated with poor loco-regional control, poor survival and treatment resistance. The advent of hybrid MRI - radiotherapy linear accelerator or 'MR Linac' systems - could permit imaging for treatment adaptation based on hypoxic status. We sought to develop oxygen-enhanced MRI (OE-MRI) in HNC and translate the technique onto an MR Linac system.

MATERIALS AND METHODS

MRI sequences were developed in phantoms and 15 healthy participants. Next, 14 HNC patients (with 21 primary or local nodal tumours) were evaluated. Baseline tissue longitudinal relaxation time (T1) was measured alongside the change in 1/T1 (termed ΔR1) between air and oxygen gas breathing phases. We compared results from 1.5 T diagnostic MR and MR Linac systems.

RESULTS

Baseline T1 had excellent repeatability in phantoms, healthy participants and patients on both systems. Cohort nasal concha oxygen-induced ΔR1 significantly increased (p < 0.0001) in healthy participants demonstrating OE-MRI feasibility. ΔR1 repeatability coefficients (RC) were 0.023-0.040 s-1 across both MR systems. The tumour ΔR1 RC was 0.013 s-1 and the within-subject coefficient of variation (wCV) was 25% on the diagnostic MR. Tumour ΔR1 RC was 0.020 s-1 and wCV was 33% on the MR Linac. ΔR1 magnitude and time-course trends were similar on both systems.

CONCLUSION

We demonstrate first-in-human translation of volumetric, dynamic OE-MRI onto an MR Linac system, yielding repeatable hypoxia biomarkers. Data were equivalent on the diagnostic MR and MR Linac systems. OE-MRI has potential to guide future clinical trials of biology guided adaptive radiotherapy.

In vivo reproducibility of 3D relayed NOE in the healthy human brain at 7 T

PURPOSE

Nuclear Overhauser effect (NOE) is based on dipolar cross-relaxation mechanism that enables the indirect detection of aliphatic protons via the water proton signal. This work focuses on determining the reproducibility of NOE magnetization transfer ratio (NOEMTR ) and isolated or relayed NOE (rNOE) contributions to the NOE MRI of the healthy human brain at 7 Tesla (T).

METHODS

We optimized theB 1 + $$ {\mathrm{B}}_1^{+} $$ amplitude and length of the saturation pulse by acquiring NOE images with differentB 1 + $$ {\mathrm{B}}_1^{+} $$ values with multiple saturation lengths. Repeated NOE MRI measurements were made on five healthy volunteers by using optimized saturation pulse parameters including correction of B0 andB 1 + $$ {\mathrm{B}}_1^{+} $$ inhomogeneities. To isolate the individual contributions from z-spectra, we have fit the NOE z-spectra using multiple Lorentzians and calculated the total contribution from each pool contributing to the overall NOEMTR contrast.

RESULTS

We found that a saturation amplitude of 0.72 μT and a length of 3 s provided the highest contrast. We found that the mean NOEMTR value in gray matter (GM) was 26%, and in white matter (WM) was 33.3% across the 3D slab of the brain. The mean rNOE contributions from GM and WM values were 8.9% and 9.6%, which were ∼10% of the corresponding total NOEMTR signal. The intersubject coefficient of variations (CoVs) of NOEMTR from GM and WM were 4.5% and 6.5%, respectively, whereas the CoVs of rNOE were 4.8% and 5.6%, respectively. The intrasubject CoVs of the NOEMTR range was 2.1%-4.2%, and rNOE range was 2.9%-10.5%.

CONCLUSION

This work has demonstrated an excellent reproducibility of both inter- and intrasubject NOEMTR and rNOE metrics in healthy human brains at 7 T.

Test-retest repeatability of ADC in prostate using the multi b-Value VERDICT acquisition

PURPOSE

VERDICT (Vascular, Extracellular, Restricted Diffusion for Cytometry in Tumours) MRI is a multi b-value, variable diffusion time DWI sequence that allows generation of ADC maps from different b-value and diffusion time combinations. The aim was to assess precision of prostate ADC measurements from varying b-value combinations using VERDICT and determine which protocol provides the most repeatable ADC.

MATERIALS AND METHODS

Forty-one men (median age: 67.7 years) from a prior prospective VERDICT study (April 2016-October 2017) were analysed retrospectively. Men who were suspected of prostate cancer and scanned twice using VERDICT were included. ADC maps were formed using 5b-value combinations and the within-subject standard deviations (wSD) were calculated per ADC map. Three anatomical locations were analysed per subject: normal TZ (transition zone), normal PZ (peripheral zone), and index lesions. Repeated measures ANOVAs showed which b-value range had the lowest wSD, Spearman correlation and generalized linear model regression analysis determined whether wSD was related to ADC magnitude and ROI size.

RESULTS

The mean lesion ADC for b0b1500 had the lowest wSD in most zones (0.18-0.58x10-4 mm2/s). The wSD was unaffected by ADC magnitude (Lesion: p = 0.064, TZ: p = 0.368, PZ: p = 0.072) and lesion Likert score (p = 0.95). wSD showed a decrease with ROI size pooled over zones (p = 0.019, adjusted regression coefficient = -1.6x10-3, larger ROIs for TZ versus PZ versus lesions). ADC maps formed with a maximum b-value of 500 s/mm2 had the largest wSDs (1.90-10.24x10-4 mm2/s).

CONCLUSION

ADC maps generated from b0b1500 have better repeatability in normal TZ, normal PZ, and index lesions.

QUANTITATIVE BIOMARKERS REPRODUCIBILITY USING GENERATIVE ADVERSARIAL APPROACHES IN REDUCED TO CONVENTIONAL DOSE CT

In recent years, several techniques for image-to-image translation by means of generative adversarial neural networks (GAN) have been proposed to learn mapping characteristics between a source and a target domain. In particular, in the medical imaging field conditional GAN frameworks with paired samples (cGAN) and unconditional cycle-consistent GANs with unpaired data (CycleGAN) have been demonstrated as a powerful scheme to model non-linear mappings that produce realistic target images from different modality sources. When proposing the usage and adaptation of these frameworks for medical image synthesis, quantitative and qualitative validation are usually performed by assessing the similarity between synthetic and target images in terms of metrics such as mean absolute error (MAE) or structural similarity (SSIM) index. However, an evaluation of clinically relevant markers showing that diagnostic information is not overlooked in the translation process is often missing. In this work, we aim at demonstrating the importance of validating medical image-to-image translation techniques by assessing their effect on the measurement of clinically relevant metrics and biomarkers. We implemented both a conditional and an unconditional approach to synthesize conventional dose chest CT scans from reduced dose CT and show that while both visually and in terms of traditional metrics the network appears to successfully minimize perceptual discrepancies, these methods are not reliable to systematically reproduce quantitative measurements of various chest biomarkers.

Repeatability and reproducibility of various 4D Flow MRI postprocessing software programs in a multi-software and multi-vendor cross-over comparison study

BACKGROUND

Different software programs are available for the evaluation of 4D Flow cardiovascular magnetic resonance (CMR). A good agreement of the results between programs is a prerequisite for the acceptance of the method. Therefore, the goal was to compare quantitative results from a cross-over comparison in individuals examined on two scanners of different vendors analyzed with four postprocessing software packages.

METHODS

Eight healthy subjects (27 ± 3 years, 3 women) were each examined on two 3T CMR systems (Ingenia, Philips Healthcare; MAGNETOM Skyra, Siemens Healthineers) with a standardized 4D Flow CMR sequence. Six manually placed aortic contours were evaluated with Caas (Pie Medical Imaging, SW-A), cvi42 (Circle Cardiovascular Imaging, SW-B), GTFlow (GyroTools, SW-C), and MevisFlow (Fraunhofer Institute MEVIS, SW-D) to analyze seven clinically used parameters including stroke volume, peak flow, peak velocity, and area as well as typically scientifically used wall shear stress values. Statistical analysis of inter- and intrareader variability, inter-software and inter-scanner comparison included calculation of absolute and relative error (ER), intraclass correlation coefficient (ICC), Bland-Altman analysis, and equivalence testing based on the assumption that inter-software differences needed to be within 80% of the range of intrareader differences.

RESULTS

SW-A and SW-C were the only software programs showing agreement for stroke volume (ICC = 0.96; ER = 3 ± 8%), peak flow (ICC: 0.97; ER = -1 ± 7%), and area (ICC = 0.81; ER = 2 ± 22%). Results from SW-A/D and SW-C/D were equivalent only for area and peak flow. Other software pairs did not yield equivalent results for routinely used clinical parameters. Especially peak maximum velocity yielded poor agreement (ICC ≤ 0.4) between all software packages except SW-A/D that showed good agreement (ICC = 0.80). Inter- and intrareader consistency for clinically used parameters was best for SW-A and SW-D (ICC = 0.56-97) and worst for SW-B (ICC = -0.01-0.71). Of note, inter-scanner differences per individual tended to be smaller than inter-software differences.

CONCLUSIONS

Of all tested software programs, only SW-A and SW-C can be used equivalently for determination of stroke volume, peak flow, and vessel area. Irrespective of the applied software and scanner, high intra- and interreader variability for all parameters have to be taken into account before introducing 4D Flow CMR in clinical routine. Especially in multicenter clinical trials a single image evaluation software should be applied.

Breaking down the RECIST 1.1 double read variability in lung trials: What do baseline assessments tell us?

Background

In clinical trials with imaging, Blinded Independent Central Review (BICR) with double reads ensures data blinding and reduces bias in drug evaluations. As double reads can cause discrepancies, evaluations require close monitoring which substantially increases clinical trial costs. We sought to document the variability of double reads at baseline, and variabilities across individual readers and lung trials.

Material and methods

We retrospectively analyzed data from five BICR clinical trials evaluating 1720 lung cancer patients treated with immunotherapy or targeted therapy. Fifteen radiologists were involved. The variability was analyzed using a set of 71 features derived from tumor selection, measurements, and disease location. We selected a subset of readers that evaluated ≥50 patients in ≥two trials, to compare individual reader’s selections. Finally, we evaluated inter-trial homogeneity using a subset of patients for whom both readers assessed the exact same disease locations. Significance level was 0.05. Multiple pair-wise comparisons of continuous variables and proportions were performed using one-way ANOVA and Marascuilo procedure, respectively.

Results

Across trials, on average per patient, target lesion (TL) number ranged 1.9 to 3.0, sum of tumor diameter (SOD) 57.1 to 91.9 mm. MeanSOD=83.7 mm. In four trials, MeanSOD of double reads was significantly different. Less than 10% of patients had TLs selected in completely different organs and 43.5% had at least one selected in different organs. Discrepancies in disease locations happened mainly in lymph nodes (20.1%) and bones (12.2%). Discrepancies in measurable disease happened mainly in lung (19.6%). Between individual readers, the MeanSOD and disease selection were significantly different (p&lt;0.001). In inter-trials comparisons, on average per patient, the number of selected TLs ranged 2.1 to 2.8, MeanSOD 61.0 to 92.4 mm. Trials were significantly different in MeanSOD (p&lt;0.0001) and average number of selected TLs (p=0.007). The proportion of patients having one of the top diseases was significantly different only between two trials for lung. Significant differences were observed for all other disease locations (p&lt;0.05).

Conclusions

We found significant double read variabilities at baseline, evidence of reading patterns and a means to compare trials. Clinical trial reliability is influenced by the interplay of readers, patients and trial design.

Improved Repeatability of Mouse Tibia Volume Segmentation in Murine Myelofibrosis Model Using Deep Learning

A murine model of myelofibrosis in tibia was used in a co-clinical trial to evaluate segmentation methods for application of image-based biomarkers to assess disease status. The dataset (32 mice with 157 3D MRI scans including 49 test-retest pairs scanned on consecutive days) was split into approximately 70% training, 10% validation, and 20% test subsets. Two expert annotators (EA1 and EA2) performed manual segmentations of the mouse tibia (EA1: all data; EA2: test and validation). Attention U-net (A-U-net) model performance was assessed for accuracy with respect to EA1 reference using the average Jaccard index (AJI), volume intersection ratio (AVI), volume error (AVE), and Hausdorff distance (AHD) for four training scenarios: full training, two half-splits, and a single-mouse subsets. The repeatability of computer versus expert segmentations for tibia volume of test-retest pairs was assessed by within-subject coefficient of variance (%wCV). A-U-net models trained on full and half-split training sets achieved similar average accuracy (with respect to EA1 annotations) for test set: AJI = 83-84%, AVI = 89-90%, AVE = 2-3%, and AHD = 0.5 mm-0.7 mm, exceeding EA2 accuracy: AJ = 81%, AVI = 83%, AVE = 14%, and AHD = 0.3 mm. The A-U-net model repeatability wCV [95% CI]: 3 [2, 5]% was notably better than that of expert annotators EA1: 5 [4, 9]% and EA2: 8 [6, 13]%. The developed deep learning model effectively automates murine bone marrow segmentation with accuracy comparable to human annotators and substantially improved repeatability.

Deep Learning for Estimating Lung Capacity on Chest Radiographs Predicts Survival in Idiopathic Pulmonary Fibrosis

Background Total lung capacity (TLC) has been estimated with use of chest radiographs based on time-consuming methods, such as planimetric techniques and manual measurements. Purpose To develop a deep learning-based, multidimensional model capable of estimating TLC from chest radiographs and demographic variables and validate its technical performance and clinical utility with use of multicenter retrospective data sets. Materials and Methods A deep learning model was pretrained with use of 50 000 consecutive chest CT scans performed between January 2015 and June 2017. The model was fine-tuned on 3523 pairs of posteroanterior chest radiographs and plethysmographic TLC measurements from consecutive patients who underwent pulmonary function testing on the same day. The model was tested with multicenter retrospective data sets from two tertiary care centers and one community hospital, including (a) an external test set 1 (n = 207) and external test set 2 (n = 216) for technical performance and (b) patients with idiopathic pulmonary fibrosis (n = 217) for clinical utility. Technical performance was evaluated with use of various agreement measures, and clinical utility was assessed in terms of the prognostic value for overall survival with use of multivariable Cox regression. Results The mean absolute difference and within-subject SD between observed and estimated TLC were 0.69 L and 0.73 L, respectively, in the external test set 1 (161 men; median age, 70 years [IQR: 61-76 years]) and 0.52 L and 0.53 L in the external test set 2 (113 men; median age, 63 years [IQR: 51-70 years]). In patients with idiopathic pulmonary fibrosis (145 men; median age, 67 years [IQR: 61-73 years]), greater estimated TLC percentage was associated with lower mortality risk (adjusted hazard ratio, 0.97 per percent; 95% CI: 0.95, 0.98; P < .001). Conclusion A fully automatic, deep learning-based model estimated total lung capacity from chest radiographs, and the model predicted survival in idiopathic pulmonary fibrosis. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Sorkness in this issue.

Repeatability of Quantitative Magnetic Resonance Imaging Biomarkers in the Tibia Bone Marrow of a Murine Myelofibrosis Model

Quantitative MRI biomarkers are sought to replace painful and invasive sequential bone-marrow biopsies routinely used for myelofibrosis (MF) cancer monitoring and treatment assessment. Repeatability of MRI-based quantitative imaging biomarker (QIB) measurements was investigated for apparent diffusion coefficient (ADC), proton density fat fraction (PDFF), and magnetization transfer ratio (MTR) in a JAK2 V617F hematopoietic transplant model of MF. Repeatability coefficients (RCs) were determined for three defined tibia bone-marrow sections (2-9 mm; 10-12 mm; and 12.5-13.5 mm from the knee joint) across 15 diseased mice from 20-37 test-retest pairs. Scans were performed on consecutive days every two weeks for a period of 10 weeks starting 3-4 weeks after transplant. The mean RC with (95% confidence interval (CI)) for these sections, respectively, were for ADC: 0.037 (0.031, 0.050), 0.087 (0.069, 0.116), and 0.030 (0.022, 0.044) μm²/ms; for PDFF: 1.6 (1.3, 2.0), 15.5 (12.5, 20.2), and 25.5 (12.0, 33.0)%; and for MTR: 0.16 (0.14, 0.19), 0.11 (0.09, 0.15), and 0.09 (0.08, 0.15). Change-trend analysis of these QIBs identified a dynamic section within the mid-tibial bone marrow in which confident changes (exceeding RC) could be observed after a four-week interval between scans across all measured MRI-based QIBs. Our results demonstrate the capability to derive quantitative imaging metrics from mouse tibia bone marrow for monitoring significant longitudinal MF changes.

Evaluation of Apparent Diffusion Coefficient Repeatability and Reproducibility for Preclinical MRIs Using Standardized Procedures and a Diffusion-Weighted Imaging Phantom

Relevant to co-clinical trials, the goal of this work was to assess repeatability, reproducibility, and bias of the apparent diffusion coefficient (ADC) for preclinical MRIs using standardized procedures for comparison to performance of clinical MRIs. A temperature-controlled phantom provided an absolute reference standard to measure spatial uniformity of these performance metrics. Seven institutions participated in the study, wherein diffusion-weighted imaging (DWI) data were acquired over multiple days on 10 preclinical scanners, from 3 vendors, at 6 field strengths. Centralized versus site-based analysis was compared to illustrate incremental variance due to processing workflow. At magnet isocenter, short-term (intra-exam) and long-term (multiday) repeatability were excellent at within-system coefficient of variance, wCV [±CI] = 0.73% [0.54%, 1.12%] and 1.26% [0.94%, 1.89%], respectively. The cross-system reproducibility coefficient, RDC [±CI] = 0.188 [0.129, 0.343] µm2/ms, corresponded to 17% [12%, 31%] relative to the reference standard. Absolute bias at isocenter was low (within 4%) for 8 of 10 systems, whereas two high-bias (>10%) scanners were primary contributors to the relatively high RDC. Significant additional variance (>2%) due to site-specific analysis was observed for 2 of 10 systems. Base-level technical bias, repeatability, reproducibility, and spatial uniformity patterns were consistent with human MRIs (scaled for bore size). Well-calibrated preclinical MRI systems are capable of highly repeatable and reproducible ADC measurements.

Multiparametric Data-driven Imaging Markers: Guidelines for Development, Application and Reporting of Model Outputs in Radiomics

This paper is the fifth in a five-part series on statistical methodology for performance assessment of multi-parametric quantitative imaging biomarkers (mpQIBs) for radiomic analysis. Radiomics is the process of extracting visually imperceptible features from radiographic medical images using data-driven algorithms. We refer to the radiomic features as data-driven imaging markers (DIMs), which are quantitative measures discovered under a data-driven framework from images beyond visual recognition but evident as patterns of disease processes irrespective of whether or not ground truth exists for the true value of the DIM. This paper aims to set guidelines on how to build machine learning models using DIMs in radiomics and to apply and report them appropriately. We provide a list of recommendations, named RANDAM (an abbreviation of "Radiomic ANalysis and DAta Modeling"), for analysis, modeling, and reporting in a radiomic study to make machine learning analyses in radiomics more reproducible. RANDAM contains five main components to use in reporting radiomics studies: design, data preparation, data analysis and modeling, reporting, and material availability. Real case studies in lung cancer research are presented along with simulation studies to compare different feature selection methods and several validation strategies.

Multiparametric Quantitative Imaging Biomarkers for Phenotype Classification: A Framework for Development and Validation

This manuscript is the third in a five-part series related to statistical assessment methodology for technical performance of multi-parametric quantitative imaging biomarkers (mp-QIBs). We outline approaches and statistical methodologies for developing and evaluating a phenotype classification model from a set of multiparametric QIBs. We then describe validation studies of the classifier for precision, diagnostic accuracy, and interchangeability with a comparator classifier. We follow with an end-to-end real-world example of development and validation of a classifier for atherosclerotic plaque phenotypes. We consider diagnostic accuracy and interchangeability to be clinically meaningful claims for a phenotype classification model informed by mp-QIB inputs, aiming to provide tools to demonstrate agreement between imaging-derived characteristics and clinically established phenotypes. Understanding that we are working in an evolving field, we close our manuscript with an acknowledgement of existing challenges and a discussion of where additional work is needed. In particular, we discuss the challenges involved with technical performance and analytical validation of mp-QIBs. We intend for this manuscript to further advance the robust and promising science of multiparametric biomarker development.

Multiparametric Quantitative Imaging in Risk Prediction: Recommendations for Data Acquisition, Technical Performance Assessment, and Model Development and Validation

Combinations of multiple quantitative imaging biomarkers (QIBs) are often able to predict the likelihood of an event of interest such as death or disease recurrence more effectively than single imaging measurements can alone. The development of such multiparametric quantitative imaging and evaluation of its fitness of use differs from the analogous processes for individual QIBs in several key aspects. A computational procedure to combine the QIB values into a model output must be specified. The output must also be reproducible and be shown to have reasonably strong ability to predict the risk of an event of interest. Attention must be paid to statistical issues not often encountered in the single QIB scenario, including overfitting and bias in the estimates of model performance. This is the fourth in a five-part series on statistical methodology for assessing the technical performance of multiparametric quantitative imaging. Considerations for data acquisition are discussed and recommendations from the literature on methodology to construct and evaluate QIB-based models for risk prediction are summarized. The findings in the literature upon which these recommendations are based are demonstrated through simulation studies. The concepts in this manuscript are applied to a real-life example involving prediction of major adverse cardiac events using automated plaque analysis.

Criteria for the translation of radiomics into clinically useful tests

Computer-extracted tumour characteristics have been incorporated into medical imaging computer-aided diagnosis (CAD) algorithms for decades. With the advent of radiomics, an extension of CAD involving high-throughput computer-extracted quantitative characterization of healthy or pathological structures and processes as captured by medical imaging, interest in such computer-extracted measurements has increased substantially. However, despite the thousands of radiomic studies, the number of settings in which radiomics has been successfully translated into a clinically useful tool or has obtained FDA clearance is comparatively small. This relative dearth might be attributable to factors such as the varying imaging and radiomic feature extraction protocols used from study to study, the numerous potential pitfalls in the analysis of radiomic data, and the lack of studies showing that acting upon a radiomic-based tool leads to a favourable benefit-risk balance for the patient. Several guidelines on specific aspects of radiomic data acquisition and analysis are already available, although a similar roadmap for the overall process of translating radiomics into tools that can be used in clinical care is needed. Herein, we provide 16 criteria for the effective execution of this process in the hopes that they will guide the development of more clinically useful radiomic tests in the future.