The validation index: a new metric for validation of segmentation algorithms using two or more expert outlines with application to radiotherapy planning

Exploring the impact of metabolic imaging in head and neck cancer treatment

Background

Target volume delineation is performed with anatomical imaging for head and neck cancer. Molecular imaging allows the recognition of specific tumor regions. Its inclusion in the pathway could lead to changes in delineation and resultant treatment plans.

Methods

PRISMA methodology was adhered to when selecting the articles for analysis and only full articles were quality assessed.

Results

Seventeen articles were included. Gross tumor volume (GTV) primary, GTV nodal, and other target volumes were evaluated. Positron emission tomography/computerized tomography (PET/CT) produced smaller primary GTVs, although not with diffusion‐weighted imaging‐magnetic resonance imaging (DWI‐MRI) or PET/MRI. The impact of these image modalities on GTV nodal did not display any consistency. Additionally, there was considerable heterogeneity in metrics comparing delineations. Four studies included appraised the dosimetric impact of the changes in target volume delineation.

Conclusion

Quantifying the impact of molecular imaging is difficult, due to heterogeneity in reporting metrics in molecular imaging modalities and a paucity of detail regarding delineation method and guideline adherence.

Effects of prenatal opioid exposure on functional networks in infancy

Prenatal opioid exposure has been linked to altered neurodevelopment and visual problems such as strabismus and nystagmus. The neural substrate underlying these alterations is unclear. Resting-state functional connectivity MRI (rsfMRI) is an advanced and well-established technique to evaluate brain networks. Few studies have examined the effects of prenatal opioid exposure on resting-state network connectivity in infancy. In this pilot study, we characterized network connectivity in opioid-exposed infants (n = 19) and controls (n = 20) between 4–8 weeks of age using both a whole-brain connectomic approach and a seed-based approach. Prenatal opioid exposure was associated with differences in distribution of betweenness centrality and connection length, with positive connections unique to each group significantly longer than common connections. The unique connections in the opioid-exposed group were more often inter-network connections while unique connections in controls and connections common to both groups were more often intra-network. The opioid-exposed group had smaller network volumes particularly in the primary visual network, but similar network strength as controls. Network topologies as determined by dice similarity index were different between groups, particularly in visual and executive control networks. These results may provide insight into the neural basis for the developmental and visual problems associated with prenatal opioid exposure.

The feasibility of using citizens to segment anatomy from medical images: Accuracy and motivation

The development of automatic methods for segmenting anatomy from medical images is an important goal for many medical and healthcare research areas. Datasets that can be used to train and test computer algorithms, however, are often small due to the difficulties in obtaining experts to segment enough examples. Citizen science provides a potential solution to this problem but the feasibility of using the public to identify and segment anatomy in a medical image has not been investigated. Our study therefore aimed to explore the feasibility, in terms of performance and motivation, of using citizens for such purposes. Public involvement was woven into the study design and evaluation. Twenty-nine citizens were recruited and, after brief training, asked to segment the spine from a dataset of 150 magnetic resonance images. Participants segmented as many images as they could within three one-hour sessions. Their accuracy was evaluated by comparing them, as individuals and as a combined consensus, to the segmentations of three experts. Questionnaires and a focus group were used to determine the citizens' motivation for taking part and their experience of the study. Citizen segmentation accuracy, in terms of agreement with the expert consensus segmentation, varied considerably between individual citizens. The citizen consensus, however, was close to the expert consensus, indicating that when pooled, citizens may be able to replace or supplement experts for generating large image datasets. Personal interest and a desire to help were the two most common reasons for taking part in the study.

LEFMIS: locally-oriented evaluation framework for medical image segmentation algorithms

This article proposes a novel framework for the locally-oriented evaluation of segmentation algorithms (LEFMIS). The presented approach is robust and takes into account local inter/intra-observer variability and the anisotropy of medical images. What is more, the framework makes it possible to distinguish types of error locally. These features are crucial in the context of cancer image data. The proposed framework is based on use of the signed anisotropic Euclidean distance transform and the distance projection. It can be used easily in many different applications with or without additional expert outlines (both inter- and intra-observer variability). The performance of the proposed framework is depicted using both artificial and kidney cancer CT data with experts' manual outlines. In the article, in the case of artificial data, it is presented that the manual outlines dispersion is symmetric in relation to the truth border. The effectiveness of the selected segmentation algorithm was analysed in the context of kidney cancer using computed tomography data. For the calculated local inter-observer variability, 80.11% of the surface points generated by the kidney segmentation algorithm are within one expert outline standard deviation and 97.96% are within five. An error distribution shift in the direction of type I error equivalent was also observed. Finally, the significance of the local estimation of error type differences is presented. The article shows the greater usefulness and flexibility of the proposed framework in comparison to the state-of-the-art methods. The exemplary usage of the LEFMIS with or without inter-/intra-observer variability is also presented.

Designing image segmentation studies: Statistical power, sample size and reference standard quality

Segmentation algorithms are typically evaluated by comparison to an accepted reference standard. The cost of generating accurate reference standards for medical image segmentation can be substantial. Since the study cost and the likelihood of detecting a clinically meaningful difference in accuracy both depend on the size and on the quality of the study reference standard, balancing these trade-offs supports the efficient use of research resources. In this work, we derive a statistical power calculation that enables researchers to estimate the appropriate sample size to detect clinically meaningful differences in segmentation accuracy (i.e. the proportion of voxels matching the reference standard) between two algorithms. Furthermore, we derive a formula to relate reference standard errors to their effect on the sample sizes of studies using lower-quality (but potentially more affordable and practically available) reference standards. The accuracy of the derived sample size formula was estimated through Monte Carlo simulation, demonstrating, with 95% confidence, a predicted statistical power within 4% of simulated values across a range of model parameters. This corresponds to sample size errors of less than 4 subjects and errors in the detectable accuracy difference less than 0.6%. The applicability of the formula to real-world data was assessed using bootstrap resampling simulations for pairs of algorithms from the PROMISE12 prostate MR segmentation challenge data set. The model predicted the simulated power for the majority of algorithm pairs within 4% for simulated experiments using a high-quality reference standard and within 6% for simulated experiments using a low-quality reference standard. A case study, also based on the PROMISE12 data, illustrates using the formulae to evaluate whether to use a lower-quality reference standard in a prostate segmentation study.

Assessment of fully-automated atlas-based segmentation of novel oral mucosal surface organ-at-risk

BACKGROUND AND PURPOSE

Current oral mucositis normal tissue complication probability models, based on the dose distribution to the oral cavity volume, have suboptimal predictive power. Improving the delineation of the oral mucosa is likely to improve these models, but is resource intensive. We developed and evaluated fully-automated atlas-based segmentation (ABS) of a novel delineation technique for the oral mucosal surfaces.

MATERIAL AND METHODS

An atlas of mucosal surface contours (MSC) consisting of 46 patients was developed. It was applied to an independent test cohort of 10 patients for whom manual segmentation of MSC structures, by three different clinicians, and conventional outlining of oral cavity contours (OCC), by an additional clinician, were also performed. Geometric comparisons were made using the dice similarity coefficient (DSC), validation index (VI) and Hausdorff distance (HD). Dosimetric comparisons were carried out using dose-volume histograms.

RESULTS

The median difference, in the DSC and HD, between automated-manual comparisons and manual-manual comparisons were small and non-significant (-0.024; p=0.33 and -0.5; p=0.88, respectively). The median VI was 0.086. The maximum normalised volume difference between automated and manual MSC structures across all of the dose levels, averaged over the test cohort, was 8%. This difference reached approximately 28% when comparing automated MSC and OCC structures.

CONCLUSIONS

Fully-automated ABS of MSC is suitable for use in radiotherapy dose-response modelling.