Interobserver variability of clinical target volume delineation in soft-tissue sarcomas

Contouring aid tools in radiotherapy. Smoothing: the false friend

OBJECTIVE

Contouring accuracy is critical in modern radiotherapy. Several tools are available to assist clinicians in this task. This study aims to evaluate the performance of the smoothing tool in the ARIA system to obtain more consistent volumes.

METHODS

Eleven different geometric shapes were delineated in ARIA v15.6 (Sphere, Cube, Square Prism, Six-Pointed Star Prism, Arrow Prism, And Cylinder and the respective volumes at 45° of axis deviation (_45)) in 1, 3, 5, 7, and 10 cm side or diameter each. Post-processing drawing tools to smooth those first-generated volumes were applied in different options (2D-ALL vs 3D) and grades (1, 3, 5, 10, 15, and 20). These volumetric transformations were analyzed by comparing different parameters: volume changes, center of mass, and DICE similarity coefficient index. Then we studied how smoothing affected two different volumes in a head and neck cancer patient: a single rounded node and the volume delineating cervical nodal areas.

RESULTS

No changes in data were found between 2D-ALL or 3D smoothing. Minimum deviations were found (range from 0 to 0.45 cm) in the center of mass. Volumes and the DICE index decreased as the degree of smoothing increased. Some discrepancies were found, especially in figures with cleft and spikes that behave differently. In the clinical case, smoothing should be applied only once throughout the target delineation process, preferably in the largest volume (PTV) to minimize errors.

CONCLUSION

Smoothing is a good tool to reduce artifacts due to the manual delineation of radiotherapy volumes. The resulting volumes must be always carefully reviewed.

The role of 18F-FDG PET in minimizing variability in gross tumor volume delineation of soft tissue sarcomas

BACKGROUND

Accurate gross tumor volume (GTV) delineation is a critical step in radiation therapy treatment planning. However, it is reader dependent and thus susceptible to intra- and inter-reader variability. GTV delineation of soft tissue sarcoma (STS) often relies on CT and MR images.

PURPOSE

This study investigates the potential role of 18F-FDG PET in reducing intra- and inter-reader variability thereby improving reproducibility of GTV delineation in STS, without incurring additional costs or radiation exposure.

MATERIALS AND METHODS

Three readers performed independent GTV delineation of 61 patients with STS using first CT and MR followed by CT, MR, and 18F-FDG PET images. Each reader performed a total of six delineation trials, three trials per imaging modality group. Dice Similarity Coefficient (DSC) score and Hausdorff distance (HD) were used to assess both intra- and inter-reader variability using generated simultaneous truth and performance level estimation (STAPLE) GTVs as ground truth. Statistical analysis was performed using a Wilcoxon signed-ranked test.

RESULTS

There was a statistically significant decrease in both intra- and inter-reader variability in GTV delineation using CT, MR 18F-FDG PET images vs. CT and MR images. This was translated by an increase in the DSC score and a decrease in the HD for GTVs drawn from CT, MR and 18F-FDG PET images vs. GTVs drawn from CT and MR for all readers and across all three trials.

CONCLUSION

Incorporation of 18F-FDG PET into CT and MR images decreased intra- and inter-reader variability and subsequently increased reproducibility of GTV delineation in STS.

Internal Guidelines for Reducing Lymph Node Contour Variability in Total Marrow and Lymph Node Irradiation

BACKGROUND

The total marrow and lymph node irradiation (TMLI) target includes the bones, spleen, and lymph node chains, with the latter being the most challenging structures to contour. We evaluated the impact of introducing internal contour guidelines to reduce the inter- and intraobserver lymph node delineation variability in TMLI treatments.

METHODS

A total of 10 patients were randomly selected from our database of 104 TMLI patients so as to evaluate the guidelines' efficacy. The lymph node clinical target volume (CTV_LN) was recontoured according to the guidelines (CTV_LN_GL_RO1) and compared to the historical guidelines (CTV_LN_Old). Both topological (i.e., Dice similarity coefficient (DSC)) and dosimetric (i.e., V95 (the volume receiving 95% of the prescription dose) metrics were calculated for all paired contours.

RESULTS

The mean DSCs were 0.82 ± 0.09, 0.97 ± 0.01, and 0.98 ± 0.02, respectively, for CTV_LN_Old vs. CTV_LN_GL_RO1, and between the inter- and intraobserver contours following the guidelines. Correspondingly, the mean CTV_LN-V95 dose differences were 4.8 ± 4.7%, 0.03 ± 0.5%, and 0.1 ± 0.1%.

CONCLUSIONS

The guidelines reduced the CTV_LN contour variability. The high target coverage agreement revealed that historical CTV-to-planning-target-volume margins were safe, even if a relatively low DSC was observed.

Feasibility study of clinical target volume definition for soft-tissue sarcoma using muscle fiber orientations derived from diffusion tensor imaging

Objective. Soft-tissue sarcoma spreads preferentially along muscle fibers. We explore the utility of deriving muscle fiber orientations from diffusion tensor MRI (DT-MRI) for defining the boundary of the clinical target volume (CTV) in muscle tissue. Approach. We recruited eight healthy volunteers to acquire MR images of the left and right thigh. The imaging session consisted of (a) two MRI spin-echo-based scans, T1- and T2-weighted; (b) a diffusion weighted (DW) spin-echo-based scan using an echo planar acquisition with fat suppression. The thigh muscles were auto-segmented using the convolutional neural network. DT-MRI data were used as a geometry encoding input to solve the anisotropic Eikonal equation with the Hamiltonian Fast-Marching method. The isosurfaces of the solution modeled the CTV boundary. Main results. The auto-segmented muscles of the thigh agreed with manually delineated with the Dice score ranging from 0.8 to 0.94 for different muscles. To validate our method of deriving muscle fiber orientations, we compared anisotropy of the isosurfaces across muscles with different anatomical orientations within a thigh, between muscles in the left and right thighs of each subject, and between different subjects. The fiber orientations were identified reproducibly across all comparisons. We identified two controlling parameters, the distance from the gross tumor volume to the isosurface and the eigenvalues ratio, to tailor the proposed CTV to the satisfaction of the clinician. Significance. Our feasibility study with healthy volunteers shows the promise of using muscle fiber orientations derived from DW MRI data for automated generation of anisotropic CTV boundary in soft tissue sarcoma. Our contribution is significant as it serves as a proof of principle for combining DT-MRI information with tumor spread modeling, in contrast to using moderately informative 2D CT planes for the CTV delineation. Such improvements will positively impact the cancer centers with a small volume of sarcoma patients.

Development of a Comprehensive, Contour-Based, Peer Review Workflow at a Community Proton Center

Purpose

Quality assurance and continuing quality improvement are integral parts of any radiation oncology practice. With increasingly conformal radiation treatments, it has become critical to focus on every slice of the target contour to ensure adequate tumor coverage and optimal normal tissue sparing. Proton therapy centers open internationally with increasing frequency, and radiation oncologists with varying degrees of subspecialization apply proton therapy in daily practice. Precise treatment with proton therapy allows us to limit toxicity but requires in-depth knowledge of the unique properties of proton beam delivery. To address this need at our proton therapy center, we developed a comprehensive peer review program to help improve the quality of care that we were providing for our patients.

Materials and Methods

We implemented a policy of comprehensive peer review for all patients treated at our community proton facility starting in January 2013. Peer review begins at the time of referral with prospective cases being reviewed for appropriateness for proton therapy at daily rounds. There is then biweekly review of target contouring and treatment plans.

Results

During a 6-month period from June 2013 to November 2013, a total of 223 new patients were treated. Documentation of peer review at chart rounds was completed for 222 of the 223 patients (99.6%). An average of 10.7 cases were reviewed in each biweekly chart rounds session, with a total of 560 case presentations. The average time required for contour review was 145 seconds (±71 seconds) and plan review was 120 seconds (±64 seconds). Modifications were suggested for 21 patients (7.9%) during contour review and for 19 patients (6.4%) during treatment plan review. An average of 4 physicians were present at each session.

Conclusions

We demonstrated that the implementation of a comprehensive, prospective peer review program is feasible in the community setting. This article can serve as a framework for future quality assurance programs.

Magnetic resonance imaging (MRI) compared with computed tomography (CT) for interobserver agreement of gross tumor volume delineation in pancreatic cancer: a multi-institutional contouring study on behalf of the AIRO group for gastrointestinal cancers

BACKGROUND

Due to the high soft tissue resolution, magnetic resonance imaging (MRI) could improve the accuracy of pancreatic tumor delineation in radiation treatment planning. A multi-institutional study was proposed to evaluate the impact of MRI on inter-observer agreement in gross tumor volume (GTV) and duodenum delineation for pancreatic cancer compared with computer tomography (CT).

MATERIAL AND METHODS

Two clinical cases of borderline resectable (Case 1) and unresectable (Case 2) pancreatic cancer were selected. In two sequential steps, diagnostic contrast-enhanced CT scan and MRI sequences were sent to the participating centers. CT-GTVs were contoured while blinded to MRI data sets. DICE index was used to evaluate the spatial overlap accuracy.

RESULTS

Thirty-one radiation oncologists from different Institutions submitted the delineated volumes. CT- and MRI-GTV mean volumes were 21.6 ± 9.0 cm³ and 17.2 ± 6.0 cm³, respectively for Case 1, and 31.3 ± 15.6 cm³ and 33.2 ± 20.2 cm³, respectively for Case 2. Resulting MRI-GTV mean volume was significantly smaller than CT-GTV in the borderline resectable case (p < .05). A substantial agreement was shown by the median DICE index for CT- and MRI-GTV resulting as 0.74 (IQR: 0.67-0.75) and 0.61 (IQR: 0.57-0.67) for Case 1; a moderate agreement was instead reported for Case 2: 0.59 (IQR:0.52-0.66) and 0.53 (IQR:0.42-0.62) for CT- and MRI-GTV, respectively.

CONCLUSION

Diagnostic MRI resulted in smaller GTV in borderline resectable case with a substantial agreement between observers, and was comparable to CT scan in interobserver variability, in both cases. The greater variability in the unresectable case underlines the critical issues related to the outlining when vascular structures are more involved. The integration of MRI with contrast-enhancement CT, thanks to its high definition of tumor relationship with neighboring vessels, could offer a greater accuracy of target delineation.

Inter-observer variability of clinical target volume delineation in definitive radiotherapy of neck lymph node metastases from unknown primary. A cooperative study of the Italian Association of Radiotherapy and Clinical Oncology (AIRO) Head and Neck Group

BACKGROUND

This study, promoted by Italian Association of Radiotherapy and Clinical Oncology (AIRO) Head and Neck Group, aimed to assess the current national practice of target volume delineation on a case of neck lymph node metastases from unknown primary evaluating inter-observer variability, in a setting of primary radiotherapy.

MATERIALS AND METHODS

A case of metastatic neck lymph node from occult primary was proposed to 17 radiation oncologists. A national reference RT center was identified and considered as benchmark. Participants were requested to delineate target volumes. A structured questionnaire was administered. A comparison between following parameters of the CTVs was performed: centroids distances, Dice similarity index (DSI), Jaccard index and mean distance to agreement (MDA). Volume expressed in cubic centimeters and CTVs cranio-caudal extension were evaluated.

RESULTS

Sixteen of 17 radiation oncologists recommended three CTVs dose levels. (CTV HD, CTV ID and CTV LD); CTV ID was not delineated by one of the participants and by the reference center. The distance between the reference centroid and the mean centroid of CTVs HD was 1.09 cm (0.36-3.99 cm); for CTV LD, a mean centroids distance of 2.45 (0.27-4.83 cm) was found, and for CTV HD, mean DSI is 0.48 and mean Jaccard index is 0.32 and MDA was 8.89 mm. CTV LD showed a mean DSI of 0.46, mean Jaccard index of 0.31 and MDA of 14.87 when compared to the reference.

CONCLUSION

Many aspects concerning treatment optimization of cervical nodes metastases from occult primary remain unclear, and we found a notable heterogeneity of global radiotherapy management reporting discordances both in target volume delineation and volume prescription.

Considerable interobserver variation in delineation of pancreatic cancer on 3DCT and 4DCT: a multi-institutional study

Background

The delineation of pancreatic tumors on CT is challenging. In this study, we quantified the interobserver variation for pancreatic tumor delineation on 3DCT as well as on 4DCT.

Methods

Eight observers (radiation oncologists) from six institutions delineated pancreatic tumors of four patients with (borderline) resectable pancreatic cancer. The study consisted of two stages. In the 3DCT-stage, the gross tumor volume (GTV) was delineated on a contrast-enhanced scan. In the 4DCT-stage, the internal GTV (iGTV) was delineated, accounting for the respiratory motion. We calculated the volumes of the (i)GTV, the overlap of the delineated volumes (expressed as generalized conformity index: CI_gen), the local observer variation (local standard deviation: SD) and the overall observer variation (overall SD). We compared these results between GTVs and iGTVs. Additionally, observers were asked to fill out a questionnaire concerning the difficulty of the delineation and their experience in delineating pancreatic tumors.

Results

The ratios of the largest to the smallest delineated GTV and iGTV within the same patient were 6.8 and 16.5, respectively. As the iGTV incorporates the GTV during all respiratory phases, the mean volumes of the iGTV (40.07 cm³) were larger than those of the GTV (29.91 cm³). For all patients, CI_gen was larger for the iGTV than for the GTV. The mean overall observer variation (root-mean-square of all local SDs over four patients) was 0.63 cm and 0.80 cm for GTV and iGTV, respectively. The largest local observer variations were seen close to biliary stents and suspicious pathological enlarged lymph nodes, as some observers included them and some did not. This variation was more pronounced for the iGTV than for the GTV. The observers rated the 3DCT-stage and 4DCT-stage equally difficult and treated on average three to four pancreatic cancer patients per year.

Conclusions

A considerable interobserver variation in delineation of pancreatic tumors was observed. This variation was larger for 4D than for 3D delineation. The largest local observer variation was found around biliary stents and suspicious pathological enlarged lymph nodes.

Electronic supplementary material

The online version of this article (doi:10.1186/s13014-017-0777-0) contains supplementary material, which is available to authorized users.

Uncertainties in volume delineation in radiation oncology: A systematic review and recommendations for future studies

BACKGROUND AND PURPOSE

Volume delineation is a well-recognised potential source of error in radiotherapy. Whilst it is important to quantify the degree of interobserver variability (IOV) in volume delineation, the resulting impact on dosimetry and clinical outcomes is a more relevant endpoint. We performed a literature review of studies evaluating IOV in target volume and organ-at-risk (OAR) delineation in order to analyse these with respect to the metrics used, reporting of dosimetric consequences, and use of statistical tests.

METHODS AND MATERIALS

Medline and Pubmed databases were queried for relevant articles using keywords. We included studies published in English between 2000 and 2014 with more than two observers.

RESULTS

119 studies were identified covering all major tumour sites. CTV (n=47) and GTV (n=38) were most commonly contoured. Median number of participants and data sets were 7 (3-50) and 9 (1-132) respectively. There was considerable heterogeneity in the use of metrics and methods of analysis. Statistical analysis of results was reported in 68% (n=81) and dosimetric consequences in 21% (n=25) of studies.

CONCLUSION

There is a lack of consistency in conducting and reporting analyses from IOV studies. We suggest a framework to use for future studies evaluating IOV.

Uncertainties in target volume delineation in radiotherapy - are they relevant and what can we do about them?

Background

Modern radiotherapy techniques enable delivery of high doses to the target volume without escalating dose to organs at risk, offering the possibility of better local control while preserving good quality of life. Uncertainties in target volume delineation have been demonstrated for most tumour sites, and various studies indicate that inconsistencies in target volume delineation may be larger than errors in all other steps of the treatment planning and delivery process. The aim of this paper is to summarize the degree of delineation uncertainties for different tumour sites reported in the literature and review the effect of strategies to minimize them.

Conclusions

Our review confirmed that interobserver variability in target volume contouring represents the largest uncertainty in the process for most tumour sites, potentially resulting in a systematic error in dose delivery, which could influence local control in individual patients. For most tumour sites the optimal combination of imaging modalities for target delineation still needs to be determined. Strict use of delineation guidelines and protocols is advisable both in every day clinical practice and in clinical studies to diminish interobserver variability. Continuing medical education of radiation oncologists cannot be overemphasized, intensive formal training on interpretation of sectional imaging should be included in the program for radiation oncology residents.

Musculoskeletal Sarcoma: Update on Imaging of the Post-treatment Patient

Post-treatment imaging of musculoskeletal sarcoma remains challenging, but newer imaging techniques are improving our ability to recognize both local and distant recurrence and accurately distinguish local recurrence from post-treatment change. We review recent advances in dynamic contrast-enhanced magnetic resonance imaging, diffusion-weighted magnetic resonance imaging with apparent diffusion coefficient mapping and positron emission tomography/computed tomography in the post-treatment follow-up of musculoskeletal sarcoma. We also describe our multidisciplinary sarcoma team approach to patient care and the essential role of the radiologist in the clinical follow-up scheme.

Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: a multi-institutional contouring experience

BACKGROUND

An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.

METHODS

A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://www.radioterapiaitalia.it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95% Confidence Interval (95% CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.

RESULTS

A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8% (95% CI: 21.2-45.0%) and 20.0% (95% CI: 14.9-30.6%) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19-0.79 in postoperative vs range 0.35-0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4% (95% CI: 30.1-72.4%) and 63.8% (95% CI: 43.9-119.2%) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.

CONCLUSIONS

The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.