Radiology Report Value Equation

Impact of an automated report comparison tool on trainee report modification rate at a tertiary hospital

AIM

This study aims to compare trainee-modified report percentage rate and trainee/consultant satisfaction regarding the feedback process before and after implementation of an automated report comparison tool.

MATERIALS AND METHODS

An automated report comparison tool utilising natural language processing, presenting the trainee's preliminary report beside the final consultant report with changes highlighted, was used in a prospective interventional study. Modification rates, including character counts, of co-authored computed tomography (CT) studies were recorded before and after tool implementation over two 6-month periods and compared with Student's t-test. Trainees and consultants were surveyed before and after the interventional period for time spent and feedback satisfaction.

RESULTS

In total, 3851 (81.7%) of 4175 reports were modified in the baseline preimplementation phase, and 5215 (69.6%) of 7489 reports were modified during the postimplementation phase (p < .001). The average character count change preimplementation was 132, corresponding to 9.0% of the original preliminary report, compared with 91 characters and 7.1% postimplementation, respectively (p < .001). This statistically significant difference generally applied regardless of the level of trainee experience. Prospective data collected in the preimplementation period revealed that for more than two-thirds of after-hours shifts, trainees spent fewer than 5 minutes receiving feedback on their after-hours work. At the conclusion of the implementation phase, 92.3% of trainees and 70% of consultants agreed that the report comparison tool improved feedback.

CONCLUSION

Following the implementation of an automated report comparison tool, there was a reduction in trainee report modification rates and subjectively improved trainee feedback. This adjunct to existing feedback mechanisms presents a relatively simple intervention to facilitate efficient case review and feedback.

Teleradiology in Saudi Arabia: a national survey and retrospective review of associated MRI reports

BACKGROUND

Due to the recent evolution of telecommunications, it is now acknowledged that digital communication provides essential services for remote areas. Teleradiology allows the ability to obtain images at one site, send them over a distance, and view them remotely for diagnostic or consultation purposes.

AIM

The highlighted objectives include (a) the added value of the service, (b) user satisfaction, and (c) quality assurance according to global best practices and national quality standards.

METHODS

This study utilised an eight-part online self-report survey distributed among employees of the Ministry of Health (MOH) who use the national teleradiology platform. The survey sections were designed to gather comprehensive data, including participant demographics, levels of satisfaction with the service, awareness of security measures, communication effectiveness, perceived advantages and disadvantages, quality assurance, technical challenges, IT support, and future perceptions of teleradiology services. Additionally, a total of 212 MRI reports from patients who underwent brain and spine MRI examinations between 2018 and 2020 were collected from the platform to strengthen the analysis.

RESULTS

Most survey respondents (78%) were males, with a significant majority (96.2%) affirming that teleradiology sufficiently addresses clinical inquiries. Furthermore, 90% expressed satisfaction with the service, and 93% endorsed the standardization of MR imaging procedures across Ministry of Health (MOH) hospitals. Notably, 92.4% recognised teleradiology as a transformative strategy for healthcare facilities in Saudi Arabia, concurring with its benefits. The analysis of the MRI reports revealed structural inconsistencies; compared with structured templates, the average number of incorporated elements was reduced, and essential elements were frequently absent. Intriguingly, reports delineating normal cases included a higher incidence of clinical impressions relative to those describing abnormalities, yet the latter contained a more comprehensive array of elements. Variability in report composition was correlated with the years of experience of the reporters. Teleradiology users perceived enhancements in the quality of radiological reporting and the daily operational workflow. Nonetheless, certain limitations were identified, necessitating focused improvements by service providers.

CONCLUSION

Despite teleradiology being a subspecialisation, it can reduce the role of local radiologists. Further research is needed on data security, confidentiality, and archiving options, as well as the cost-effectiveness of teleradiology services.

AI-Supported Comprehensive Detection and Quantification of Biomarkers of Subclinical Widespread Diseases at Chest CT for Preventive Medicine

Automated image analysis plays an increasing role in radiology in detecting and quantifying image features outside of the perception of human eyes. Common AI-based approaches address a single medical problem, although patients often present with multiple interacting, frequently subclinical medical conditions. A holistic imaging diagnostics tool based on artificial intelligence (AI) has the potential of providing an overview of multi-system comorbidities within a single workflow. An interdisciplinary, multicentric team of medical experts and computer scientists designed a pipeline, comprising AI-based tools for the automated detection, quantification and characterization of the most common pulmonary, metabolic, cardiovascular and musculoskeletal comorbidities in chest computed tomography (CT). To provide a comprehensive evaluation of each patient, a multidimensional workflow was established with algorithms operating synchronously on a decentralized Joined Imaging Platform (JIP). The results of each patient are transferred to a dedicated database and summarized as a structured report with reference to available reference values and annotated sample images of detected pathologies. Hence, this tool allows for the comprehensive, large-scale analysis of imaging-biomarkers of comorbidities in chest CT, first in science and then in clinical routine. Moreover, this tool accommodates the quantitative analysis and classification of each pathology, providing integral diagnostic and prognostic value, and subsequently leading to improved preventive patient care and further possibilities for future studies.

The natural language processing of radiology requests and reports of chest imaging: Comparing five transformer models’ multilabel classification and a proof-of-concept study

Background

Radiology requests and reports contain valuable information about diagnostic findings and indications, and transformer-based language models are promising for more accurate text classification.

Methods

In a retrospective study, 2256 radiologist-annotated radiology requests (8 classes) and reports (10 classes) were divided into training and testing datasets (90% and 10%, respectively) and used to train 32 models. Performance metrics were compared by model type (LSTM, Bertje, RobBERT, BERT-clinical, BERT-multilingual, BERT-base), text length, data prevalence, and training strategy. The best models were used to predict the remaining 40,873 cases’ categories of the datasets of requests and reports.

Results

The RobBERT model performed the best after 4000 training iterations, resulting in AUC values ranging from 0.808 [95% CI (0.757–0.859)] to 0.976 [95% CI (0.956–0.996)] for the requests and 0.746 [95% CI (0.689–0.802)] to 1.0 [95% CI (1.0–1.0)] for the reports. The AUC for the classification of normal reports was 0.95 [95% CI (0.922–0.979)]. The predicted data demonstrated variability of both diagnostic yield for various request classes and request patterns related to COVID-19 hospital admission data.

Conclusion

Transformer-based natural language processing is feasible for the multilabel classification of chest imaging request and report items. Diagnostic yield varies with the information in the requests.

Comparison of the clinical usefulness of structured and free-text reports for interpretation of jaw lesions on cone beam computed tomography images

OBJECTIVE

This study compared the clinical usefulness of structured reports (SRs) and free-text reports (FTRs) of lesions depicted on cone beam computed tomography (CBCT) images from the perspectives of report providers and receivers.

STUDY DESIGN

In total, 36 CBCT images of jaw lesions obtained between February 2020 and August 2020 were evaluated. A working group of 3 oral and maxillofacial radiologists (OMRs) established a reporting system and prepared reports. Evaluation group I (2 OMRs) wrote SRs and FTRs for each case and assessed the reporting process for the criteria of convenience and organization. Evaluation group II (3 general practitioners [GPs] and 3 oral and maxillofacial surgeons [OMSs]) assessed the reports for the criteria of productivity, consistency, and organization. A 5-point Likert scale was used to assess the usefulness of each report. Scores were statistically compared according to report type with the paired Wilcoxon signed-rank test.

RESULTS

The SRs scored significantly higher for all criteria as assessed by evaluation group I and the GPs of group II (P < .001). The FTRs scored significantly higher for productivity and organization as assessed by the OMSs of group II (P = .005 for both criteria).

CONCLUSIONS

The clinical usefulness of reports may differ according to roles of the report recipients in diagnosis and treatment.

Talking Points: Enhancing Communication Between Radiologists and Patients

Radiologists communicate along multiple pathways, using written, verbal, and non-verbal means. Radiology trainees must gain skills in all forms of communication, with attention to developing effective professional communication in all forms. This manuscript reviews evidence-based strategies for enhancing effective communication between radiologists and patients through direct communication, written means and enhanced reporting. We highlight patient-centered communication efforts, available evidence, and opportunities to engage learners and enhance training and simulation efforts that improve communication with patients at all levels of clinical care.

Adoption of a diagnostic certainty scale in abdominal imaging: 2-year experience at an academic institution

PURPOSE

Assess use of a diagnostic certainty scale (CS) for abdominal imaging reports and identify factors associated with greater adoption.

METHODS

This retrospective, Institutional Review Board-exempt study was conducted at an academic health system. Abdominal radiology reports containing diagnostic certainty phrases (DCPs) generated 4/1/2019-3/31/2021 were identified by a natural language processing tool. Reports containing DCPs were subdivided into those with/without a CS inserted at the end. Primary outcome was monthly CS use rate in reports containing DCPs. Secondary outcomes were assessment of factors associated with CS use, and usage of recommended DCPs over time. Chi-square test was used to compare proportions; univariable and multivariable regression assessed impact of other variables.

RESULTS

DCPs were used in 81,281/124,501 reports (65.3%). One-month post-implementation, 82/2310 (3.6%) of reports with DCPs contained the CS, increasing to 1862/4644 (40.1%) by study completion (p < 0.001). Multivariable analysis demonstrated reports containing recommended DCPs were more likely to have the CS (Odds Ratio [OR] 4.5; p < 0.001). Using CT as a reference, CS use was lower for ultrasound (OR 0.73; p < 0.001) and X-ray (OR 0.38; p < 0.001). There was substantial inter-radiologist variation in CS use (OR 0.01-26.3, multiple p values).

CONCLUSION

DCPs are very common in abdominal imaging reports and can be further clarified with CS use. Although voluntary CS adoption increased 13-fold over 2 years, > 50% of reports with DCPs lacked the CS at study's end. More stringent interventions, including embedding the scale in report templates, are likely needed to reduce inter-radiologist variation and decrease ambiguity in conveying diagnostic certainty to referring providers and patients.

Patient-centered Reporting in Radiology: A Single-site Survey Study of Lung Cancer Screening Results

PURPOSE

This study aimed to assess whether patients preferred traditional or patient-friendly radiology reports and, secondarily, whether one reporting style led to a subjective improvement in patients' understanding of their imaging results and next steps in their clinical care.

MATERIALS AND METHODS

This randomized study included patients who had previously enrolled in an institutional comprehensive lung cancer screening program. Three hundred patients were randomly selected from the program database to receive both traditional and patient-centered radiology reports. Randomization also occurred at both the risk level of the fictitious test results (low, intermediate, or high) and the order in which the reports were read by each participant. Participants completed a survey providing demographic information and indicating which report style was preferred and which report style led to a better understanding of screening results and future options. In addition, each report style was rated (from 1 to 5) for clarity, understandability, attractiveness, and helpfulness.

RESULTS

A total of 46 responses for report preference data and 41 responses for attribute rating data were obtained. Overall, participants demonstrate a preference for patient-friendly reports (65.2%) over traditional reports (21.7%). On a 5-point scale, average ratings for patient-friendly reports were higher than traditional reports by 1.2 (P<0.001) for clarity, 1.5 (P<0.001) for understandability, 1.5 (P<0.001) for attractiveness, and 1.0 (P<0.001) for helpfulness.

CONCLUSION

Data suggest that patients prefer patient-friendly reports over traditional reports and find them to be clearer, more comprehensible, more attractive, and more helpful.

Artificial Intelligence for Quality Improvement in Radiology

Artificial intelligence (AI) and informatics promise to improve the quality and efficiency of diagnostic radiology but will require substantially more standardization and operational coordination to realize and measure those improvements. As radiology steps into the AI-driven future we should work hard to identify the needs and desires of our customers and develop process controls to ensure we are meeting them. Rather than focusing on easy-to-measure turnaround times as surrogates for quality, AI and informatics can support more comprehensive quality metrics, such as ensuring that reports are accurate, readable, and useful to patients and health care providers.

Namiri N, Shah R, Bharadwaj U, M. Link T, D. Bucknor M, Pedoia V, Majumdar S. Automatic Deep Learning-assisted Detection and Grading of Abnormalities in Knee MRI Studies

PURPOSE

To test the hypothesis that artificial intelligence (AI) techniques can aid in identifying and assessing lesion severity in the cartilage, bone marrow, meniscus, and anterior cruciate ligament (ACL) in the knee, improving overall MRI interreader agreement.

MATERIALS AND METHODS

This retrospective study was conducted on 1435 knee MRI studies (n = 294 patients; mean age, 43 years ± 15 [standard deviation]; 153 women) collected within three previous studies (from 2011 to 2014). All MRI studies were acquired using high-spatial-resolution three-dimensional fast-spin-echo CUBE sequence. Three-dimensional convolutional neural networks were developed to detect the regions of interest within MRI studies and grade abnormalities of the cartilage, bone marrow, menisci, and ACL. Evaluation included sensitivity, specificity, and Cohen linear-weighted ĸ. The impact of AI-aided grading in intergrader agreement was assessed on an external dataset.

RESULTS

Binary lesion sensitivity reported for all tissues was between 70% and 88%. Specificity ranged from 85% to 89%. The area under the receiver operating characteristic curve for all tissues ranged from 0.83 to 0.93. Deep learning-assisted intergrader Cohen ĸ agreement significantly improved in 10 of 16 comparisons among two attending physicians and two trainees for all tissues.

CONCLUSION

The three-dimensional convolutional neural network had high sensitivity, specificity, and accuracy for knee-lesion-severity scoring and also increased intergrader agreement when used on an external dataset.Supplemental material is available for this article. Keywords: Bone Marrow, Cartilage, Computer Aided Diagnosis (CAD), Computer Applications-3D, Computer Applications-Detection/Diagnosis, Knee, Ligaments, MR-Imaging, Neural Networks, Observer Performance, Segmentation, Statistics © RSNA, 2021See also the commentary by Li and Chang in this issue.: An earlier incorrect version of this article appeared online. This article was corrected on April 16, 2021.

Improving Billing Accuracy Through Enterprise-Wide Standardized Structured Reporting With Cross-Divisional Shared Templates

OBJECTIVE

We describe our experience in implementing enterprise-wide standardized structured reporting for chest radiographs (CXRs) via change management strategies and assess the economic impact of structured template adoption.

METHODS

Enterprise-wide standardized structured CXR reporting was implemented in a large urban health care enterprise in two phases from September 2016 to March 2019: initial implementation of division-specific structured templates followed by introduction of auto launching cross-divisional consensus structured templates. Usage was tracked over time, and potential radiologist time savings were estimated. Correct-to-bill (CTB) rates were collected between January 2018 and May 2019 for radiography.

RESULTS

CXR structured template adoption increased from 46% to 92% in phase 1 and to 96.2% in phase 2, resulting in an estimated 8.5 hours per month of radiologist time saved. CTB rates for both radiographs and all radiology reports showed a linearly increasing trend postintervention with radiography CTB rate showing greater absolute values with an average difference of 20% throughout the sampling period. The CTB rate for all modalities increased by 12%, and the rate for radiography increased by 8%.

DISCUSSION

Change management strategies prompted adoption of division-specific structured templates, and exposure via auto launching enforced widespread adoption of consensus templates. Standardized structured reporting resulted in both economic gains and projected radiologist time saved.

Discrepant Reporting Style Preferences Between Clinicians and Radiologists

RATIONALE AND OBJECTIVES

To compare preferences in reporting styles between radiologists and clinicians in structured vs unstructured reporting styles in order to facilitate better communication.

METHODS

An online survey was distributed to 5280 clinicians, radiologists, and physicians in training surveying respondent preference for three different reporting styles: expanded structured, minimized structured, and unstructured.

RESULTS

A 7.5% response rate was achieved. Overall, the expanded structured reporting style was the most preferred (47%, 186/394). This contrasted with radiologists who preferred the unstructured reporting style (41%), whereas nonradiologists preferred the expanded structured reporting style (51%; P < 0.001). There was significance in emergency medicine physicians preferring the minimized structured reporting style (51%, 27/43), whereas all other specialties preferred the expanded structured report (49%, 168/341; P = 0.0038).

DISCUSSION

There is a discrepant reporting style preference between clinicians and radiologists. A structured reporting style with expanded standard statements is preferred by most physicians. Radiologists could consider using a structured reporting style with minimized normal statements in the emergency room setting.

Reporting on Renal Masses, Recommendations for Terminology, and Sample Templates

Given the incidence of small renal masses, from benign cysts to malignancy, most radiologists encounter these lesions multiple times during their career. Radiologists have an opportunity to provide critical data that will further refine the understanding of the impact of these masses on patient outcomes. This article summarizes and describes recent updates and understanding of the critical observations and descriptors of renal masses. The templates and glossary of terms presented in this review article facilitate the radiology reporting of such data elements, giving radiologists the opportunity to improve diagnostic accuracy and influence management of small renal masses.