Statistical evaluation of reader variability in assessing the diagnostic performance of optical coherence tomography

SIGNIFICANCE

Optical coherence tomography (OCT) is widely used as a potential diagnostic tool for a variety of diseases including various types of cancer. However, sensitivity and specificity analyses of OCT in different cancers yield results varying from 11% to 100%. Hence, there is a need for more detailed statistical analysis of blinded reader studies.

AIM

Extensive statistical analysis is performed on results from a blinded study involving OCT of breast tumor margins to assess the impact of reader variability on sensitivity and specificity.

APPROACH

Five readers with varying levels of experience reading OCT images assessed 50 OCT images of breast tumor margins collected using an intraoperative OCT system. Statistical modeling and analysis was performed using the R language to analyze reader experience and variability.

RESULTS

Statistical analysis showed that the readers' prior experience with OCT images was directly related to the probability of the readers' assessment agreeing with histology. Additionally, results from readers with prior experience specific to OCT in breast cancer had a higher probability of agreement with histology compared to readers with experience with OCT in other (noncancer) diseases.

CONCLUSIONS

The results from this study demonstrate the potential impact of reader training and experience in the assessment of sensitivity and specificity. They also demonstrate even greater potential improvement in diagnostic performance by combining results from multiple readers. These preliminary findings suggest valuable directions for further study.

Intraoperative optical coherence tomography for assessing human lymph nodes for metastatic cancer

Background

Evaluation of lymph node (LN) status is an important factor for detecting metastasis and thereby staging breast cancer. Currently utilized clinical techniques involve the surgical disruption and resection of lymphatic structure, whether nodes or axillary contents, for histological examination. While reasonably effective at detection of macrometastasis, the majority of the resected lymph nodes are histologically negative. Improvements need to be made to better detect micrometastasis, minimize or eliminate lymphatic disruption complications, and provide immediate and accurate intraoperative feedback for in vivo cancer staging to better guide surgery.

Methods

We evaluated the use of optical coherence tomography (OCT), a high-resolution, real-time, label-free imaging modality for the intraoperative assessment of human LNs for metastatic disease in patients with breast cancer. We assessed the sensitivity and specificity of double-blinded trained readers who analyzed intraoperative OCT LN images for presence of metastatic disease, using co-registered post-operative histopathology as the gold standard.

Results

Our results suggest that intraoperative OCT examination of LNs is an appropriate real-time, label-free, non-destructive alternative to frozen-section analysis, potentially offering faster interpretation and results to empower superior intraoperative decision-making.

Conclusions

Intraoperative OCT has strong potential to supplement current post-operative histopathology with real-time in situ assessment of LNs to preserve both non-cancerous nodes and their lymphatic vessels, and thus reduce the associated risks and complications from surgical disruption of lymphoid structures following biopsy.

Real-time Imaging of the Resection Bed Using a Handheld Probe to Reduce Incidence of Microscopic Positive Margins in Cancer Surgery

Wide local excision (WLE) is a common surgical intervention for solid tumors such as those in melanoma, breast, pancreatic, and gastrointestinal cancer. However, adequate margin assessment during WLE remains a significant challenge, resulting in surgical reinterventions to achieve adequate local control. Currently, no label-free imaging method is available for surgeons to examine the resection bed in vivo for microscopic residual cancer. Optical coherence tomography (OCT) enables real-time high-resolution imaging of tissue microstructure. Previous studies have demonstrated that OCT analysis of excised tissue specimens can distinguish between normal and cancerous tissues by identifying the heterogeneous and disorganized microscopic tissue structures indicative of malignancy. In this translational study involving 35 patients, a handheld surgical OCT imaging probe was developed for in vivo use to assess margins both in the resection bed and on excised specimens for the microscopic presence of cancer. The image results from OCT showed structural differences between normal and cancerous tissue within the resection bed following WLE of the human breast. The ex vivo images were compared with standard postoperative histopathology to yield sensitivity of 91.7% [95% confidence interval (CI), 62.5%-100%] and specificity of 92.1% (95% CI, 78.4%-98%). This study demonstrates in vivo OCT imaging of the resection bed during WLE with the potential for real-time microscopic image-guided surgery.

Noninvasive Surface Imaging of Breast Cancer in Humans using a Hand-held Optical Imager

X-ray mammography, the current gold standard for breast cancer detection, has a 20% false-negative rate (cancer is undetected) and increases in younger women with denser breast tissue. Diffuse optical imaging (DOI) is a safe (nonionizing), and relatively inexpensive method for noninvasive imaging of breast cancer in human subjects (including dense breast tissues) by providing physiological information (e.g. oxy- and deoxy- hemoglobin concentration). At the Optical Imaging Laboratory, a hand-held optical imager has been developed which employs a breast contourable probe head to perform simultaneous illumination and detection of large surfaces towards near real-time imaging of human breast cancer. Gen-1 and gen-2 versions of the handheld optical imager have been developed and previously demonstrated imaging in tissue phantoms and healthy human subjects. Herein, the hand-held optical imagers are applied towards in vivo imaging of breast cancer subjects in an attempt to determine the ability of the imager to detect breast tumors. Five female human subjects (ages 51-74) diagnosed with breast cancer were imaged with the gen-1 optical imager prior to surgical intervention. One of the subjects was also imaged with the gen-2 optical imager. Both imagers use 785 nm laser diode sources and ICCD camera detectors to generate 2D surfaces maps of total hemoglobin absorption. The subjects lay in supine position and images were collected at various locations on both the ipsilateral (tumor-containing) and contralateral (non-tumor containing) breasts. The optical images (2D surface maps of optical absorption due to total hemoglobin concentration) show regions of higher intensity at the tumor location, which is indicative of increased vasculature and higher blood content due to the presence of the tumor. Additionally, a preliminary result indicates the potential to image lymphatic spread. This study demonstrates the potential of the hand-held optical devices to noninvasively image breast cancer in human subjects.

Abstract P2-03-11: In situ imaging of the tumor cavity during breast lumpectomy using optical coherence tomography

Re-operation rates for breast lumpectomy procedures are exceedingly high, often over 30%, depending on the institution and surgical technique. Because current standard-of-care relies on post-operative histopathology to provide a microscopic view and assessment of surgical margins, there has been great interest in developing new imaging solutions to visualize tissues intraoperatively with high-resolution, and provide real-time feedback on the margin status. While it is possible to use a variety of microscopic imaging methods in the operating suite, including frozen-section histology, touch-prep cytology, confocal or scattering-based microscopy, all these techniques are limited to visualizing margins on ex vivo resected specimens, and do not provide a means for visualizing the in situ tumor cavity for evidence of positive margins or residual disease.

Optical coherence tomography (OCT) is a high-resolution, real-time, optical biomedical imaging technology that is the optical analogue to ultrasound imaging, except images are based on backscattered near-infrared light. OCT is capable of performing optical biopsies of in situ tissue at resolutions that approach those in histopathology. With the use of an advanced computed imaging technique called ISAM (Interferometric Synthetic Aperture Microscopy), even higher imaging resolution over larger depths is possible, commensurate with the depths (1-2 mm) visualized by pathologists to determine negative, close, or positive margins. Past studies by our group and others have demonstrated the feasibility of intraoperative OCT for assessing tumor margin and lymph node status during breast cancer surgery, but to date, all studies have been performed on resected lumpectomy tissue.

In this study, we report the development of a novel handheld surgical imaging probe that enables 2-D and 3-D OCT/ISAM imaging of the in situ tumor cavity, in addition to the margins of excised specimens. To date, this handheld OCT/ISAM probe has been used in 10 breast cancer surgeries where both in situ and ex vivo imaging was performed. Four of these cases involved in situ imaging of the cavity margin after a suspicious area was visually and tactically identified, and was subsequently resected, followed by ex vivo imaging and validating post-operative histopathology. Representative cases included fibroadipose tissue, fibroadenomas, and high-grade ductal carcinoma in situ.

Distinct microstructural features identified on OCT/ISAM and confirmed with histopathology demonstrate that this technique can visualize the in situ tumor cavity, as well as the surgical margins on resected specimens, with micron-scale resolution. OCT/ISAM has the potential to determine margin status in real-time during the surgical procedure, when further surgical resection to establish clear margins and reduce re-operation rates is possible.

Citation Information: Cancer Res 2013;73(24 Suppl): Abstract nr P2-03-11.

Abstract P1-01-23: Intraoperative optical coherence tomography for the assessment of metastatic disease in human lymph nodes

The status of lymph nodes is an important factor in staging cancer since the transport of primary cancer cells via the lymphatic system is one of the main pathways of metastasis to distant organs. During cancer surgery, lymph node status is evaluated via sentinel lymph node biopsy (SLNB), which involves the removal and analysis of the first (or sentinel) nodes along the lymphatic chain of nodes draining the primary tumor. The sentinel nodes are identified through the accumulation of a radioactive agent (technetium-99) and/or isosulfan blue dye within the nodes, frequently resulting in the resection of multiple nodes for subsequent, often time-consuming, histopathological analysis. The majority of these resected nodes are found by histological analysis to be normal, leading to unnecessary complications, including increased risk of lymphedema. Thus, a method for the in situ assessment of node status could reduce the number of normal lymph nodes that are resected. In this study we evaluated the sensitivity and specificity of three-dimensional optical coherence tomography (OCT) for the intraoperative assessment of metastatic disease in lymph nodes.

OCT is the optical analogue to ultrasound imaging, except images are based on the optical scattering properties of near-infrared light. Real-time OCT with micron-scale resolution affords optical biopsies of tissue for immediate feedback. Intraoperative OCT imaging was conducted on human lymph nodes resected from 49 subjects during breast and, head and neck cancer surgeries. Three-dimensional OCT datasets were recorded ex vivo from one or more locations per node, and marked with surgical ink for subsequent correlation to histology. These lymph nodes then underwent the standard histological processing.

A double-blinded study was performed comparing the assessment of OCT datasets to the co-registered histological findings. Three-dimensional -OCT datasets from 206 sites were independently analyzed by six observers and classified as cancerous or non-cancerous. Seventy-nine sets were identified as unsuitable for OCT analysis due to insufficient nodal tissue within the imaged field-of-view. Early training classification results from three of the six observers resulted in a sensitivity of 64.8% and a specificity of 73.3% for identifying metastatic lymph nodes intraoperatively, in real-time, compared to the gold standard of post-operative histopathology. Final study results are expected to improve with observer training and a decision tree for interpreting OCT images.

Our initial imaging studies of resected lymph nodes in human cancer subjects demonstrate the potential of OCT as a technique for real-time optical biopsy of lymph nodes for the intraoperative staging of cancer.

Citation Information: Cancer Res 2013;73(24 Suppl): Abstract nr P1-01-23.