Improving radiologist's ability in identifying particular abnormal lesions on mammograms through training test set with immediate feedback

Surgical and Radiology Trainees' Proficiency in Reading Mammograms: the Importance of Education for Cancer Localisation

Medical imaging with mammography plays a very important role in screening and diagnosis of breast cancer, Australia's most common female cancer. The visualisation of cancers on mammograms often forms a diagnosis and guidance for radiologists and breast surgeons, and education platforms that provide real cases in a simulated testing environment have been shown to improve observer performance for radiologists. This study reports on the performance of surgical and radiology trainees in locating breast cancers. An enriched test set of 20 mammography cases (6 cancer and 14 cancer free) was created, and 18 surgical trainees and 32 radiology trainees reviewed the cases via the Breast Screen Reader Assessment Strategy (BREAST) platform and marked any lesions identifiable. Further analysis of performance with high- and low-density cases was undertaken, and standard metrics including sensitivity and specificity. Radiology trainees performed significantly better than surgical trainees in terms of specificity (0.72 vs. 0.35; P < 0.01). No significant differences were observed between the surgical and radiology trainees in sensitivity or lesion sensitivity. Mixed results were obtained with participants regarding breast density, with higher density cases generally having lower performance. The higher specificity of the radiology trainees compared to the surgical trainees likely represents less exposure to negative mammography cases. The use of high-fidelity simulated self-test environments like BREAST is able to benchmark, understand and build strategies for improving cancer education in a safe environment, including identifying challenging scenarios like breast density for enhanced training.

Global Radiomic Features from Mammography for Predicting Difficult-To-Interpret Normal Cases

This work aimed to investigate whether global radiomic features (GRFs) from mammograms can predict difficult-to-interpret normal cases (NCs). Assessments from 537 readers interpreting 239 normal mammograms were used to categorise cases as 120 difficult-to-interpret and 119 easy-to-interpret based on cases having the highest and lowest difficulty scores, respectively. Using lattice- and squared-based approaches, 34 handcrafted GRFs per image were extracted and normalised. Three classifiers were constructed: (i) CC and (ii) MLO using the GRFs from corresponding craniocaudal and mediolateral oblique images only, based on the random forest technique for distinguishing difficult- from easy-to-interpret NCs, and (iii) CC + MLO using the median predictive scores from both CC and MLO models. Useful GRFs for the CC and MLO models were recognised using a scree test. The CC and MLO models were trained and validated using the leave-one-out-cross-validation. The models' performances were assessed by the AUC and compared using the DeLong test. A Kruskal-Wallis test was used to examine if the 34 GRFs differed between difficult- and easy-to-interpret NCs and if difficulty level based on the traditional breast density (BD) categories differed among 115 low-BD and 124 high-BD NCs. The CC + MLO model achieved higher performance (0.71 AUC) than the individual CC and MLO model alone (0.66 each), but statistically non-significant difference was found (all p > 0.05). Six GRFs were identified to be valuable in describing difficult-to-interpret NCs. Twenty features, when compared between difficult- and easy-to-interpret NCs, differed significantly (p < 0.05). No statistically significant difference was observed in difficulty between low- and high-BD NCs (p = 0.709). GRF mammographic analysis can predict difficult-to-interpret NCs.

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Mammography interpretation is challenging with high error rates. This study aims to reduce the errors in mammography reading by mapping diagnostic errors against global mammographic characteristics using a radiomics-based machine learning approach. A total of 36 radiologists from cohort A (n = 20) and cohort B (n = 16) read 60 high-density mammographic cases. Radiomic features were extracted from three regions of interest (ROIs), and random forest models were trained to predict diagnostic errors for each cohort. Performance was evaluated using sensitivity, specificity, accuracy, and AUC. The impact of ROI placement and normalization on prediction was investigated. Our approach successfully predicted both the false positive and false negative errors of both cohorts but did not consistently predict location errors. The errors produced by radiologists from cohort B were less predictable compared to those in cohort A. The performance of the models did not show significant improvement after feature normalization, despite the mammograms being produced by different vendors. Our novel radiomics-based machine learning pipeline focusing on global radiomic features could predict false positive and false negative errors. The proposed method can be used to develop group-tailored mammographic educational strategies to help improve future mammography reader performance.

The Impact of Prior Mammograms on the Diagnostic Performance of Radiologists in Early Breast Cancer Detection: A Focus on Breast Density, Lesion Features and Vendors Using Wholly Digital Screening Cases

BACKGROUND

This study aims to investigate the diagnostic efficacy of radiologists when reading screening mammograms in the absence of previous images, and with the presence of prior images from the same and different vendors.

METHODS

612 radiologists' readings across 9 test sets, consisting of 540 screening mammograms (361-normal and 179-cancer) with 245 cases having prior images obtained from same vendor as current images, 129 from a different vendor and 166 cases having no prior images, were retrospectively analysed. True positive (sensitivity), true negative (specificity) and area under ROC curve (AUC) values of radiologists were calculated for three groups of cases (without prior images (NP), with prior images from same vendor (SP), and with prior images from different vendor (DP)). Logistic regression was used to estimate the odds ratio (OR) of true positive, true negative and true cancer localization among case groups with different levels of breast density and lesion characteristics.

RESULTS

Radiologists obtained 12.8% and 10.3% higher sensitivity in NP and DP than SP (0.803-and-0.785 vs. 0.712; p < 0.0001). Specificity in NP and DP cases were 4.8% and 2.0% lower than SP cases (0.749 and 0.771 vs. 0.787). The AUC values for NP and DP were significantly higher than SP cases across different levels of breast density (0.814-and-0.820 vs. 0.782; p < 0.0001). The odds ratio (OR) of true positive for NP relative to SP was 1.6 (p < 0.0001) and DP relative to SP was 1.5 (p < 0.0001). Radiologists were more like to detect architectural distortion in DP than SP cases (OR = 3.2; p < 0.0001), whilst the OR for abnormal calcifications was 2.85 (p < 0.0001).

CONCLUSIONS

Cases without previous mammograms or with prior mammograms obtained from different vendors were more likely to benefit radiologists in cancer detection, whilst prior mammograms undertaken from the same vendor were more useful for radiologists in evaluating normal cases.

Look how far we have come: BREAST cancer detection education on the international stage

The development of screening mammography over 30 years has remarkedly reduced breast cancer-associated mortality by 20%-30% through detection of small cancer lesions at early stages. Yet breast screening programmes may function differently in each nation depending on the incidence rate, national legislation, local health infrastructure and training opportunities including feedback on performance. Mammography has been the frontline breast cancer screening tool for several decades; however, it is estimated that there are 15% to 35% of cancers missed on screening which are owing to perceptual and decision-making errors by radiologists and other readers. Furthermore, mammography screening is not available in all countries and the increased speed in the number of new breast cancer cases among less developed countries exceeds that of the developed world in recent decades. Studies conducted through the BreastScreen Reader Assessment Strategy (BREAST) training tools for breast screening readers have documented benchmarking and significant variation in diagnostic performances in screening mammogram test sets in different countries. The performance of the radiologists from less well-established breast screening countries such as China, Mongolia and Vietnam were significant lower in detecting early-stage cancers than radiologists from developed countries such as Australia, USA, Singapore, Italy. Differences in breast features and cancer presentations, discrepancies in the level of experiences in reading screening mammograms, the availability of high-quality national breast screening program and breast image interpretation training courses between developed and less developed countries are likely to have impact on the variation of readers' performances. Hence dedicated education training programs with the ability to tailor to different reader cohorts and different population presentations are suggested to ameliorate challenges in exposure to a range of cancer cases and improve the interpretation skills of local radiologists. Findings from this review provide a good understanding of the radiologist' performances and their improvement using the education interventions, primarily the BREAST program, which has been deployed in a large range of developing and developed countries in the last decade. Self-testing and immediate feedback loops have been shown to have important implications for benchmarking and improving the diagnostic accuracy in radiology worldwide for better breast cancer control.

Learning effects in visual grading assessment of model-based reconstruction algorithms in abdominal Computed Tomography

Objectives

Images reconstructed with higher strengths of iterative reconstruction algorithms may impair radiologists' subjective perception and diagnostic performance due to changes in the amplitude of different spatial frequencies of noise. The aim of the present study was to ascertain if radiologists can learn to adapt to the unusual appearance of images produced by higher strengths of Advanced modeled iterative reconstruction algorithm (ADMIRE).

Methods

Two previously published studies evaluated the performance of ADMIRE in non-contrast and contrast-enhanced abdominal CT. Images from 25 (first material) and 50 (second material) patients, were reconstructed with ADMIRE strengths 3, 5 (AD3, AD5) and filtered back projection (FBP). Radiologists assessed the images using image criteria from the European guidelines for quality criteria in CT. To ascertain if there was a learning effect, new analyses of data from the two studies was performed by introducing a time variable in the mixed-effects ordinal logistic regression model.

Results

In both materials, a significant negative attitude to ADMIRE 5 at the beginning of the viewing was strengthened during the progress of the reviews for both liver parenchyma (first material: -0.70, p < 0.01, second material: -0.96, p < 0.001) and overall image quality (first material:-0.59, p < 0.05, second material::-1.26, p < 0.001). For ADMIRE 3, an early positive attitude for the algorithm was noted, with no significant change over time for all criteria except one (overall image quality), where a significant negative trend over time (-1.08, p < 0.001) was seen in the second material.

Conclusions

With progression of reviews in both materials, an increasing dislike for ADMIRE 5 images was apparent for two image criteria. In this time perspective (weeks or months), no learning effect towards accepting the algorithm could be demonstrated.