Interobserver concordance in the BI-RADS classification of breast ultrasound exams

Breast Ultrasound Volume Sweep Imaging: A New Horizon in Expanding Imaging Access for Breast Cancer Detection

OBJECTIVE

The majority of people in the world lack basic access to breast diagnostic imaging resulting in delay to diagnosis of breast cancer. In this study, we tested a volume sweep imaging (VSI) ultrasound protocol for evaluation of palpable breast lumps that can be performed by operators after minimal training without prior ultrasound experience as a means to increase accessibility to breast ultrasound.

METHODS

Medical students without prior ultrasound experience were trained for less than 2 hours on the VSI breast ultrasound protocol. Patients presenting with palpable breast lumps for standard of care ultrasound examination were scanned by a trained medical student with the VSI protocol using a Butterfly iQ handheld ultrasound probe. Video clips of the VSI scan imaging were later interpreted by an attending breast imager. Results of VSI scan interpretation were compared to the same-day standard of care ultrasound examination.

RESULTS

Medical students scanned 170 palpable lumps with the VSI protocol. There was 97% sensitivity and 100% specificity for a breast mass on VSI corresponding to 97.6% agreement with standard of care (Cohen's κ = 0.95, P < .0001). There was a detection rate of 100% for all cancer presenting as a sonographic mass. High agreement for mass characteristics between VSI and standard of care was observed, including 87% agreement on Breast Imaging-Reporting and Data System assessments (Cohen's κ = 0.82, P < .0001).

CONCLUSIONS

Breast ultrasound VSI for palpable lumps offers a promising means to increase access to diagnostic imaging in underserved areas. This approach could decrease delay to diagnosis for breast cancer, potentially improving morbidity and mortality.

No sonographer, no radiologist: Assessing accuracy of artificial intelligence on breast ultrasound volume sweep imaging scans

Breast ultrasound provides a first-line evaluation for breast masses, but the majority of the world lacks access to any form of diagnostic imaging. In this pilot study, we assessed the combination of artificial intelligence (Samsung S-Detect for Breast) with volume sweep imaging (VSI) ultrasound scans to evaluate the possibility of inexpensive, fully automated breast ultrasound acquisition and preliminary interpretation without an experienced sonographer or radiologist. This study was conducted using examinations from a curated data set from a previously published clinical study of breast VSI. Examinations in this data set were obtained by medical students without prior ultrasound experience who performed VSI using a portable Butterfly iQ ultrasound probe. Standard of care ultrasound exams were performed concurrently by an experienced sonographer using a high-end ultrasound machine. Expert-selected VSI images and standard of care images were input into S-Detect which output mass features and classification as "possibly benign" and "possibly malignant." Subsequent comparison of the S-Detect VSI report was made between 1) the standard of care ultrasound report by an expert radiologist, 2) the standard of care ultrasound S-Detect report, 3) the VSI report by an expert radiologist, and 4) the pathological diagnosis. There were 115 masses analyzed by S-Detect from the curated data set. There was substantial agreement of the S-Detect interpretation of VSI among cancers, cysts, fibroadenomas, and lipomas to the expert standard of care ultrasound report (Cohen's κ = 0.73 (0.57-0.9 95% CI), p<0.0001), the standard of care ultrasound S-Detect interpretation (Cohen's κ = 0.79 (0.65-0.94 95% CI), p<0.0001), the expert VSI ultrasound report (Cohen's κ = 0.73 (0.57-0.9 95% CI), p<0.0001), and the pathological diagnosis (Cohen's κ = 0.80 (0.64-0.95 95% CI), p<0.0001). All pathologically proven cancers (n = 20) were designated as "possibly malignant" by S-Detect with a sensitivity of 100% and specificity of 86%. Integration of artificial intelligence and VSI could allow both acquisition and interpretation of ultrasound images without a sonographer and radiologist. This approach holds potential for increasing access to ultrasound imaging and therefore improving outcomes related to breast cancer in low- and middle- income countries.

Enhancing Performance of Breast Ultrasound in Opportunistic Screening Women by a Deep Learning-Based System: A Multicenter Prospective Study

PURPOSE

To validate the feasibility of S-Detect, an ultrasound computer-aided diagnosis (CAD) system using deep learning, in enhancing the diagnostic performance of breast ultrasound (US) for patients with opportunistic screening-detected breast lesions.

METHODS

Nine medical centers throughout China participated in this prospective study. Asymptomatic patients with US-detected breast masses were enrolled and received conventional US, S-Detect, and strain elastography subsequently. The final pathological results are referred to as the gold standard for classifying breast mass. The diagnostic performances of the three methods and the combination of S-Detect and elastography were evaluated and compared, including sensitivity, specificity, and area under the receiver operating characteristics (AUC) curve. We also compared the diagnostic performances of S-Detect among different study sites.

RESULTS

A total of 757 patients were enrolled, including 460 benign and 297 malignant cases. S-Detect exhibited significantly higher AUC and specificity than conventional US (AUC, S-Detect 0.83 [0.80-0.85] vs. US 0.74 [0.70-0.77], p < 0.0001; specificity, S-Detect 74.35% [70.10%-78.28%] vs. US 54.13% [51.42%-60.29%], p < 0.0001), with no decrease in sensitivity. In comparison to that of S-Detect alone, the AUC value significantly was enhanced after combining elastography and S-Detect (0.87 [0.84-0.90]), without compromising specificity (73.93% [68.60%-78.78%]). Significant differences in the S-Detect's performance were also observed across different study sites (AUC of S-Detect in Groups 1-4: 0.89 [0.84-0.93], 0.84 [0.77-0.89], 0.85 [0.76-0.92], 0.75 [0.69-0.80]; p [1 vs. 4] < 0.0001, p [2 vs. 4] = 0.0165, p [3 vs. 4] = 0.0157).

CONCLUSIONS

Compared with the conventional US, S-Detect presented higher overall accuracy and specificity. After S-Detect and strain elastography were combined, the performance could be further enhanced. The performances of S-Detect also varied among different centers.

Positive Predictive Value for the Malignancy of Mammographic Abnormalities Based on the Presence of an Ultrasound Correlate

Purpose To compare the outcomes of different mammographic lesions based on the presence of an ultrasound (US) correlate and to estimate how often targeted US can identify such lesions.

Materials and Methods This retrospective study included all consecutive cases from 2010 to 2016, with Breast Imaging Reporting and Database System (BI-RADS) categories 4 & 5 who underwent US as part of their diagnostic workup. We compared the incidence of malignancy between lesions comprising a US correlate that underwent US-guided core needle biopsy (CNB) and those without a correlate that underwent stereotactic CNB.

Results 833 lesions met the study criteria and included masses (64.3%), architectural distortion (19%), asymmetries (4.6%), and calcifications (12.1%). The CNB-based positive predictive value (PPV) was higher for lesions with a US correlate than for those without (40.2% [36.1, 44.4%] vs. 18.9% [14.5, 23.9%], respectively) (p<0.001). Malignancy odds for masses, asymmetries, architectural distortion, and calcifications were greater by 2.70, 4.17, 4.98, and 2.77 times, respectively, for the US-guided CNB (p<0.001, p=0.091, p<0.001, and p=0.034, respectively). Targeted US identified a correlate to 66.3% of the mammographic findings. The odds of finding a correlate were greater for masses (77.8%) than architectural distortions (53.8%) (p<0.001) or calcifications (24.8%) (p<0.001).

Conclusion The success of targeted US in identifying a correlate varies significantly according to the type of mammographic lesion. The PPV of lesions with a US correlate was significantly higher than that of those with no correlate. However, the PPV of lesions with no US correlate is high enough (18.9%) to warrant a biopsy.

Comparison of Automated Breast Ultrasound and Hand-Held Breast Ultrasound in the Screening of Dense Breasts

OBJECTIVE

 To compare hand-held breast ultrasound (HHBUS) and automated breast ultrasound (ABUS) as screening tool for cancer.

METHODS

 A cross-sectional study in patients with mammographically dense breasts was conducted, and both HHBUS and ABUS were performed. Hand-held breast ultrasound was acquired by radiologists and ABUS by mammography technicians and analyzed by breast radiologists. We evaluated the Breast Imaging Reporting and Data System (BI-RADS) classification of the exam and of the lesion, as well as the amount of time required to perform and read each exam. The statistical analysis employed was measures of central tendency and dispersion, frequencies, Student t test, and a univariate logistic regression, through the odds ratio and its respective 95% confidence interval, and with p < 0.05 considered of statistical significance.

RESULTS

 A total of 440 patients were evaluated. Regarding lesions, HHBUS detected 15 (7.7%) BI-RADS 2, 175 (89.3%) BI-RADS 3, and 6 (3%) BI-RADS 4, with 3 being confirmed by biopsy as invasive ductal carcinomas (IDCs), and 3 false-positives. Automated breast ultrasound identified 12 (12.9%) BI-RADS 2, 75 (80.7%) BI-RADS 3, and 6 (6.4%) BI-RADS 4, including 3 lesions detected by HHBUS and confirmed as IDCs, in addition to 1 invasive lobular carcinoma and 2 high-risk lesions not detected by HHBUS. The amount of time required for the radiologist to read the ABUS was statistically inferior compared with the time required to read the HHBUS (p < 0.001). The overall concordance was 80.9%. A total of 219 lesions were detected, from those 70 lesions by both methods, 126 only by HHBUS (84.9% not suspicious by ABUS) and 23 only by ABUS.

CONCLUSION

 Compared with HHBUS, ABUS allowed adequate sonographic study in supplemental screening for breast cancer in heterogeneously dense and extremely dense breasts.

Preliminary study of the technical limitations of automated breast ultrasound: from procedure to diagnosis

Objective

To evaluate the main technical limitations of automated breast ultrasound and to determine the proportion of examinations excluded.

Materials and Methods

We evaluated 440 automated breast ultrasound examinations performed, over a 12-month period, by technicians using an established protocol.

Results

In five cases (1.1%), the examination was deemed unacceptable for diagnostic purposes, those examinations therefore being excluded.

Conclusion

Automated breast ultrasound is expected to overcome some of the major limitations of conventional ultrasound in breast cancer screening. In Brazil, this new method can be accepted for inclusion in routine clinical practice only after its advantages have been validated in the national context.

Interobserver agreement in breast ultrasound categorization in the Mammography and Ultrasonography Study for Breast Cancer Screening Effectiveness (MUST-BE) trial: results of a preliminary study

Purpose

The purpose of this study was to record and evaluate interobserver agreement as quality control for the modified categorization of screening breast ultrasound developed by the Alliance for Breast Cancer Screening in Korea (ABCS-K) for the Mammography and Ultrasonography Study for Breast Cancer Screening Effectiveness (MUST-BE) trial.

Methods

Eight breast radiologists with 4-16 years of experience participated in 2 rounds of quality control testing for the MUST-BE trial. Two investigators randomly selected 125 and 100 cases of breast lesions with different ratios of malignant and benign lesions. Two versions of the modified categorization were tested. The initially modified classification was developed after the first quality control workshop, and the re-modified classification was developed after the second workshop. The re-modified categorization established by ABCS-K added size criteria and the anterior-posterior ratio compared with the initially modified classification. After a brief lecture on the modified categorization system prior to each quality control test, the eight radiologists independently categorized the lesions using the modified categorization. Interobserver agreement was measured using kappa statistics.

Results

The overall kappa values for the modified categorizations indicated moderate to substantial degrees of agreement (initially modified categorization and re-modified categorization: κ=0.52 and κ=0.63, respectively). The kappa values for the subcategories of category 4 were 0.37 (95% confidence interval [CI], 0.24 to 0.52) and 0.39 (95% CI, 0.31 to 0.49), respectively. The overall kappa values for both the initially modified categorization and the re-modified categorization indicated a substantial degree of agreement when dichotomizing the interpretation as benign or suspicious.

Conclusion

The preliminary results demonstrated acceptable interobserver agreement for the modified categorization.

Variations in the sonographic measurement techniques of BI-RADS 3 breast masses

OBJECTIVE

To evaluate the differences in sonographic (US) distance and volume measurements from different sonologists and identify the optimal parameters to avoid clinically relevant variations in the measurement of BI-RADS 3 breast masses.

METHODS

For this cross-sectional study with prospectively collected data, four physicians with various levels of experience in US, herein called sonologists, performed distance and volume US measurements of 80 masses classified as BI-RADS 3. The Cochran Q test was used to compare the matched sets of rates of clinically relevant variability between all pairs of sonologists' measurements.

RESULTS

There were clinically relevant differences between sonologists in the measurements of the longest diameter (range, 17.5-43.7%, p = 0.003), the longest diameter perpendicular to the previous one (anteroposterior diameter) (17.5-33.7%, p = 0.06), the third diameter orthogonal to the plane defined by the previous two (transverse diameter) (28.7-55%, p = 0.001), and at least two of those three diameters (18.7-38.7%, p = 0.015). The smallest clinically relevant differences were observed with volume measurements (range of differences, 6.2-13.7%, p = 0.51).

CONCLUSIONS

Volume measurement technique was associated with the least variations, whereas distance measurements, which are used routinely, were associated with unacceptable rates of clinically relevant variations. © 2016 Wiley Periodicals, Inc. J Clin Ultrasound 45:252-260, 2017.

Contrast-enhanced ultrasound improved performance of breast imaging reporting and data system evaluation of critical breast lesions

AIM: To determine whether contrast-enhanced ultrasound (CEUS) can improve the precision of breast imaging reporting and data system (BI-RADS) categorization.

METHODS: A total of 230 patients with 235 solid breast lesions classified as BI-RADS 4 on conventional ultrasound were evaluated. CEUS was performed within one week before core needle biopsy or surgical resection and a revised BI-RADS classification was assigned based on 10 CEUS imaging characteristics. Receiver operating characteristic curve analysis was then conducted to evaluate the diagnostic performance of CEUS-based BI-RADS assignment with pathological examination as reference criteria.

RESULTS: The CEUS-based BI-RADS evaluation classified 116/235 (49.36%) lesions into category 3, 20 (8.51%), 13 (5.53%) and 12 (5.11%) lesions into categories 4A, 4B and 4C, respectively, and 74 (31.49%) into category 5. Selecting CEUS-based BI-RADS category 4A as an appropriate cut-off gave sensitivity and specificity values of 85.4% and 87.8%, respectively, for the diagnosis of malignant disease. The cancer-to-biopsy yield was 73.11% with CEUS-based BI-RADS 4A selected as the biopsy threshold compared with 40.85% otherwise, while the biopsy rate was only 42.13% compared with 100% otherwise. Overall, only 4.68% of invasive cancers were misdiagnosed.

CONCLUSION: This pilot study suggests that evaluation of BI-RADS 4 breast lesions with CEUS results in reduced biopsy rates and increased cancer-to-biopsy yields.

Variability in Observer Performance Between Faculty Members and Residents Using Breast Imaging Reporting and Data System (BI-RADS)-Ultrasound, Fifth Edition (2013)

Background

Ultrasonography (US) is a useful tool for breast imaging, yet is highly operator-dependent.

Objectives

We evaluated inter-observer variability and performance discrepancies between faculty members and radiology residents when describing breast lesions, by the fifth edition of breast imaging reporting and data system (BI-RADS)-US lexicon, and then attempted to identify whether inter-observer variability could be improved after one education session.

Patients and Methods

In total, 50 malignant lesions and 70 benign lesions were considered in our retrospective study. Two faculty members, two senior residents, and two junior residents separately assessed the US images. After the first assessment, the readers received one education session, and then reassessed the images in a random order. Inter-observer variability was measured using the kappa coefficient (κ). Performance discrepancy was evaluated by receiver operating characteristic (ROC) curves.

Results

For the faculty members, fair-to-good agreement was obtained in all descriptors and final assessment, while for residents, poor-to-moderate agreement was obtained. The areas under the ROC curves were 0.78 for the faculty members, 0.59 for the senior residents, and 0.52 for the junior residents, respectively. Diagnostic performance was significantly higher in the faculty members than the senior and junior residents (P = 0.0001 and < 0.0001, respectively). After one education session, the agreement in the final assessment was one level higher in the faculty members and senior residents, yet in the senior residents, the degree of agreement was still only fair. Moreover, in the junior residents, there was no improvement.

Conclusion

Investigative assessment of breast US by residents is inadvisable. We recommend continued professional resident training to improve the degree of agreement and performance.