Use of Radiology Procedure Codes in Health Care: The Need for Standardization and Structure

Radiology procedure codes are a fundamental part of most radiology workflows, such as ordering, scheduling, billing, and image interpretation. Nonstandardized unstructured procedure codes have typically been used in radiology departments. Such codes may be sufficient for specific purposes, but they offer limited support for interoperability. As radiology workflows and the various forms of clinical data exchange have become more sophisticated, the need for more advanced interoperability with use of standardized structured codes has increased. For example, structured codes facilitate the automated identification of relevant prior imaging studies and the collection of data for radiation dose tracking. The authors review the role of imaging procedure codes in radiology departments and across the health care enterprise. Standards for radiology procedure coding are described, and the mechanisms of structured coding systems are reviewed. In particular, the structure of the RadLex™ Playbook coding system and examples of the use of this system are described. Harmonization of the RadLex Playbook system with the Logical Observation Identifiers Names and Codes standard, which is currently in progress, also is described. The benefits and challenges of adopting standardized codes-especially the difficulties in mapping local codes to standardized codes-are reviewed. Tools and strategies for mitigating these challenges, including the use of billing codes as an intermediate step in mapping, also are reviewed. In addition, the authors describe how to use the RadLex Playbook Web service application programming interface for partial automation of code mapping. ^© RSNA, 2017.

Deep Learning to Classify Radiology Free-Text Reports

Purpose To evaluate the performance of a deep learning convolutional neural network (CNN) model compared with a traditional natural language processing (NLP) model in extracting pulmonary embolism (PE) findings from thoracic computed tomography (CT) reports from two institutions. Materials and Methods Contrast material-enhanced CT examinations of the chest performed between January 1, 1998, and January 1, 2016, were selected. Annotations by two human radiologists were made for three categories: the presence, chronicity, and location of PE. Classification of performance of a CNN model with an unsupervised learning algorithm for obtaining vector representations of words was compared with the open-source application PeFinder. Sensitivity, specificity, accuracy, and F1 scores for both the CNN model and PeFinder in the internal and external validation sets were determined. Results The CNN model demonstrated an accuracy of 99% and an area under the curve value of 0.97. For internal validation report data, the CNN model had a statistically significant larger F1 score (0.938) than did PeFinder (0.867) when classifying findings as either PE positive or PE negative, but no significant difference in sensitivity, specificity, or accuracy was found. For external validation report data, no statistical difference between the performance of the CNN model and PeFinder was found. Conclusion A deep learning CNN model can classify radiology free-text reports with accuracy equivalent to or beyond that of an existing traditional NLP model. ^© RSNA, 2017 Online supplemental material is available for this article.

Performance of a Deep-Learning Neural Network Model in Assessing Skeletal Maturity on Pediatric Hand Radiographs

Purpose To compare the performance of a deep-learning bone age assessment model based on hand radiographs with that of expert radiologists and that of existing automated models. Materials and Methods The institutional review board approved the study. A total of 14 036 clinical hand radiographs and corresponding reports were obtained from two children's hospitals to train and validate the model. For the first test set, composed of 200 examinations, the mean of bone age estimates from the clinical report and three additional human reviewers was used as the reference standard. Overall model performance was assessed by comparing the root mean square (RMS) and mean absolute difference (MAD) between the model estimates and the reference standard bone ages. Ninety-five percent limits of agreement were calculated in a pairwise fashion for all reviewers and the model. The RMS of a second test set composed of 913 examinations from the publicly available Digital Hand Atlas was compared with published reports of an existing automated model. Results The mean difference between bone age estimates of the model and of the reviewers was 0 years, with a mean RMS and MAD of 0.63 and 0.50 years, respectively. The estimates of the model, the clinical report, and the three reviewers were within the 95% limits of agreement. RMS for the Digital Hand Atlas data set was 0.73 years, compared with 0.61 years of a previously reported model. Conclusion A deep-learning convolutional neural network model can estimate skeletal maturity with accuracy similar to that of an expert radiologist and to that of existing automated models. ^© RSNA, 2017 An earlier incorrect version of this article appeared online. This article was corrected on January 19, 2018.

Clinical decision support increases diagnostic yield of computed tomography for suspected pulmonary embolism

OBJECTIVE

Determine effects of evidence-based clinical decision support (CDS) on the use and yield of computed tomographic pulmonary angiography for suspected pulmonary embolism (CTPE) in Emergency Department (ED) patients.

METHODS

This multi-site prospective quality improvement intervention conducted in three urban EDs used a pre/post design. For ED patients aged 18+years with suspected PE, CTPE use and yield were compared 19months pre- and 32months post-implementation of CDS intervention based on the Wells criteria, provided at the time of CTPE order, deployed in April 2012. Primary outcome was the yield (percentage of studies positive for acute PE). Secondary outcome was utilization (number of studies/100 ED visits) of CTPE. Chi-square and statistical process control chart assessed pre- and post-intervention differences. An interrupted time series analysis was also performed.

RESULTS

Of 558,795 patients presenting October 2010-December 2014, 7987 (1.4%) underwent CTPE (mean age 52±17.5years, 66% female, 60.1% black); 34.7% of patients presented pre- and 65.3% post-CDS implementation. Overall CTPE diagnostic yield was 9.8% (779/7987 studies positive for PE). Yield increased a relative 30.8% after CDS implementation (8.1% vs. 10.6%; p=0.0003). There was no statistically significant change in CTPE utilization (1.4% pre- vs. 1.4% post-implementation; p=0.25). A statistical process control chart demonstrated immediate and sustained improvement in CTPE yield post-implementation. Interrupted time series analysis demonstrated the slope of PE findings versus time to be unchanged before and after the intervention (p=0.9). However, there was a trend that the intervention was associated with a 50% increased probability of PE finding (p=0.08), suggesting an immediate rather than gradual change after the intervention.

CONCLUSIONS

Implementing evidence-based CDS in the ED was associated with an immediate, significant and sustained increase in CTPE yield without a measurable decrease in CTPE utilization. Further studies will be needed to assess whether stronger interventions could further improve appropriate use of CTPE.

Implications of Direct Patient Online Access to Radiology Reports Through Patient Web Portals

In an era of increasing health information transparency and informed decision making, more patients are being provided with direct online access to their medical records, including radiology reports, via web-based portals. Although radiologists' narrative reports have previously been the purview of referring physicians, patients are now reading these on their own. Many potential benefits may result from patients reviewing their radiology reports, including improvements in patients' own understanding of their health, promotion of shared decision making and patient-physician communication, and, ultimately, improvements in patient outcomes. However, there may also be negative consequences, including confusion and anxiety among patients and longer patient-physician interactions. The rapid adoption of this new technology has led to major questions regarding ethics and professionalism for radiologists, including the following: Who is the intended audience of radiology reports? How should content be presented or worded? How will open access influence radiologists' relationships with patients and referring physicians? What legal ramifications may arise from increased patient access? The authors describe the current practices and research findings associated with patient online access to medical records, including radiology reports, and discuss several implications of this growing trend for the radiology profession.

Unsupervised Topic Modeling in a Large Free Text Radiology Report Repository

Radiology report narrative contains a large amount of information about the patient's health and the radiologist's interpretation of medical findings. Most of this critical information is entered in free text format, even when structured radiology report templates are used. The radiology report narrative varies in use of terminology and language among different radiologists and organizations. The free text format and the subtlety and variations of natural language hinder the extraction of reusable information from radiology reports for decision support, quality improvement, and biomedical research. Therefore, as the first step to organize and extract the information content in a large multi-institutional free text radiology report repository, we have designed and developed an unsupervised machine learning approach to capture the main concepts in a radiology report repository and partition the reports based on their main foci. In this approach, radiology reports are modeled in a vector space and compared to each other through a cosine similarity measure. This similarity is used to cluster radiology reports and identify the repository's underlying topics. We applied our approach on a repository of 1,899,482 radiology reports from three major healthcare organizations. Our method identified 19 major radiology report topics in the repository and clustered the reports accordingly to these topics. Our results are verified by a domain expert radiologist and successfully explain the repository's primary topics and extract the corresponding reports. The results of our system provide a target-based corpus and framework for information extraction and retrieval systems for radiology reports.

"Chasing a Ghost": Factors that Influence Primary Care Physicians to Follow Up on Incidental Imaging Findings

Purpose To explore provider and patient characteristics that influence how primary care providers (PCPs) communicate and manage incidental imaging findings. Materials and Methods This HIPAA-compliant study was approved by the institutional review board. Through semistructured interviews, researchers explored concerns and perspectives of 30 PCPs on receiving and acting on incidental imaging findings. Open-ended questions were designed to elicit a range of responses rather than quantifiable data. Thematic codes were developed and explicitly defined. Three research assistants independently coded all 30 deidentified transcripts and resolved discrepancies (κ = 0.85). Codes pertaining to PCP and patient characteristics were organized into an explanatory model. Results Some PCPs felt compelled but frustrated to pursue costly follow-up for incidental imaging findings of limited clinical importance. Other PCPs did not act on findings that were unfamiliar or occurred in an unusual clinical context when follow-up recommendations were not given; the challenges of researching the clinical importance of these findings or seeking specialist consultation led to inaction. Some PCPs reported using a uniform approach to communicate and manage incidental findings, while others adapted their approach to the patient and the finding. Sometimes PCP characteristics such as follow-up style superseded patient characteristics. At other times patient characteristics such as health literacy superseded PCP characteristics. Conclusion PCPs cited a variety of objective and subjective factors that influence how they communicate and manage incidental imaging findings. These results suggest that some patients may receive inappropriate follow-up of incidental imaging findings and present an opportunity for radiologists to help PCPs and patients to best use the information conveyed in imaging reports. ^© RSNA, 2016 Online supplemental material is available for this article.

Predicting High Imaging Utilization Based on Initial Radiology Reports: A Feasibility Study of Machine Learning

RATIONALE AND OBJECTIVES

Imaging utilization has significantly increased over the last two decades, and is only recently showing signs of moderating. To help healthcare providers identify patients at risk for high imaging utilization, we developed a prediction model to recognize high imaging utilizers based on their initial imaging reports.

MATERIALS AND METHODS

The prediction model uses a machine learning text classification framework. In this study, we used radiology reports from 18,384 patients with at least one abdomen computed tomography study in their imaging record at Stanford Health Care as the training set. We modeled the radiology reports in a vector space and trained a support vector machine classifier for this prediction task. We evaluated our model on a separate test set of 4791 patients. In addition to high prediction accuracy, in our method, we aimed at achieving high specificity to identify patients at high risk for high imaging utilization.

RESULTS

Our results (accuracy: 94.0%, sensitivity: 74.4%, specificity: 97.9%, positive predictive value: 87.3%, negative predictive value: 95.1%) show that a prediction model can enable healthcare providers to identify in advance patients who are likely to be high utilizers of imaging services.

CONCLUSIONS

Machine learning classifiers developed from narrative radiology reports are feasible methods to predict imaging utilization. Such systems can be used to identify high utilizers, inform future image ordering behavior, and encourage judicious use of imaging.

Information extraction from multi-institutional radiology reports

OBJECTIVES

The radiology report is the most important source of clinical imaging information. It documents critical information about the patient's health and the radiologist's interpretation of medical findings. It also communicates information to the referring physicians and records that information for future clinical and research use. Although efforts to structure some radiology report information through predefined templates are beginning to bear fruit, a large portion of radiology report information is entered in free text. The free text format is a major obstacle for rapid extraction and subsequent use of information by clinicians, researchers, and healthcare information systems. This difficulty is due to the ambiguity and subtlety of natural language, complexity of described images, and variations among different radiologists and healthcare organizations. As a result, radiology reports are used only once by the clinician who ordered the study and rarely are used again for research and data mining. In this work, machine learning techniques and a large multi-institutional radiology report repository are used to extract the semantics of the radiology report and overcome the barriers to the re-use of radiology report information in clinical research and other healthcare applications.

MATERIAL AND METHODS

We describe a machine learning system to annotate radiology reports and extract report contents according to an information model. This information model covers the majority of clinically significant contents in radiology reports and is applicable to a wide variety of radiology study types. Our automated approach uses discriminative sequence classifiers for named-entity recognition to extract and organize clinically significant terms and phrases consistent with the information model. We evaluated our information extraction system on 150 radiology reports from three major healthcare organizations and compared its results to a commonly used non-machine learning information extraction method. We also evaluated the generalizability of our approach across different organizations by training and testing our system on data from different organizations.

RESULTS

Our results show the efficacy of our machine learning approach in extracting the information model's elements (10-fold cross-validation average performance: precision: 87%, recall: 84%, F1 score: 85%) and its superiority and generalizability compared to the common non-machine learning approach (p-value<0.05).

CONCLUSIONS

Our machine learning information extraction approach provides an effective automatic method to annotate and extract clinically significant information from a large collection of free text radiology reports. This information extraction system can help clinicians better understand the radiology reports and prioritize their review process. In addition, the extracted information can be used by researchers to link radiology reports to information from other data sources such as electronic health records and the patient's genome. Extracted information also can facilitate disease surveillance, real-time clinical decision support for the radiologist, and content-based image retrieval.

True "meaningful use": technology meets both patient and provider needs

OBJECTIVES

Voluntary patient uptake and use of electronic health record (EHR) features have been low. It is unknown whether EHRs fully meet needs of providers or patients with chronic diseases.

STUDY DESIGN

To explore in-depth user experiences, we conducted 6 focus groups: 3 of patients followed by 3 of providers discussing 2 key EHR components: the after-visit summary (AVS) and the patient portal (PP). Focus groups were audio-recorded, transcribed, and analyzed by 3 independent coders.

METHODS

Participants with moderate-to-severe asthma and prevalent comorbidities were recruited from 4 primary care and 2 asthma clinics serving low-income urban neighborhoods. Participants discussed their expectations and experience using the AVS and PP, and responded to prototype formats of these features. Additionally, one-on-one interviews were conducted with 10 patients without PP experience to assess their ability to use the system.

RESULTS

The 21 patient and 13 provider perspectives differed regarding AVS features and use. Patients wanted a unified view of their medical issues and health management tools, while providers wanted to focus on recommendations from 1 visit at a time. Both groups advocated improving the AVS format and content. Lack of awareness and knowledge about the PP was patients' largest barrier, and was traced back to providers' lack of PP training.

CONCLUSIONS

Our results underscore the importance of user-centered design when constructing the content and features of the EHR. As technology evolves, an ongoing understanding of patient and provider experiences will be critical to improve uptake, increase use, and ensure engagement, optimizing the potential of EHRs.

Assessment of follow-up completeness and notification preferences for imaging findings of possible cancer: what happens after radiologists submit their reports?

RATIONALE AND OBJECTIVES

To understand the reasons leading to potentially inappropriate management of imaging findings concerning for malignancy and identify optimal methods for communicating these findings to providers.

MATERIALS AND METHODS

We identified all abdominal imaging examinations with findings of possible cancer performed on six randomly selected days in August to December 2013. Electronic medical records (EMR) of one patient group were reviewed 3 months after the index examination to determine whether management was appropriate (completed follow-up or documented reason for no follow-up) or potentially inappropriate (no follow-up or no documented reason). Providers of a second patient group were contacted 5-6 days after imaging examinations to determine notification preferences.

RESULTS

Among 43 patients in the first group, five (12%) received potentially inappropriate management. Reasons included patient loss to follow-up and provider failure to review imaging results, document known imaging findings, or communicate findings to providers outside the health system. Among 16 providers caring for patients in the second group, 33% were unaware of the findings, 75% preferred to be notified of abnormal findings via e-mail or EMR, 56% wanted an embedded hyperlink enabling immediate follow-up order entry, and only 25% had a system to monitor whether patients had completed ordered testing.

CONCLUSIONS

One in eight patients did not receive potentially necessary follow-up care within 3 months of imaging findings of possible cancer. Automated notification of imaging findings and follow-up monitoring not only is desired by providers but can also address many of the reasons we found for inappropriate management.

Clinical decision support for imaging in the era of the Patient Protection and Affordable Care Act

Imaging clinical decision support (CDS) systems provide evidence for or against imaging procedures ordered within a computerized physician order entry system at the time of the image order. Depending on the pertinent clinical history provided by the ordering clinician, CDS systems can optimize imaging by educating providers on appropriate image order entry and by alerting providers to the results of prior, potentially relevant imaging procedures, thereby reducing redundant imaging. The American Recovery and Reinvestment Act (ARRA) has expedited the adoption of computerized physician order entry and CDS systems in health care through the creation of financial incentives and penalties to promote the "meaningful use" of health IT. Meaningful use represents the latest logical next step in a long chain of legislation promoting the areas of appropriate imaging utilization, accurate reporting, and IT. It is uncertain if large-scale implementation of imaging CDS will lead to improved health care quality, as seen in smaller settings, or to improved patient outcomes. However, imaging CDS enables the correlation of existing imaging evidence with outcome measures, including morbidity, mortality, and short-term imaging-relevant management outcomes (eg, biopsy, chemotherapy). The purposes of this article are to review the legislative sequence relevant to imaging CDS and to give guidance to radiology practices focused on quality and financial performance improvement during this time of accelerating regulatory change.

Extracting templates from radiology reports using sequence alignment

Health care providers often dictate their reports by filling in slots in templates. These slots can be filled with a variety of different procedures, measurements, or findings. Many radiologists currently create their own personalised templates, costing time and leading to inconsistencies across physicians. We present a sequence alignment method Radiology Content Alignment (RADICAL) that uses dynamic programming to efficiently extract templates that are common across sets of reports, and give examples of the extracted templates and the contents of their slots.

Automated detection of critical results in radiology reports

The goal of this study was to develop and validate text-mining algorithms to automatically identify radiology reports containing critical results including tension or increasing/new large pneumothorax, acute pulmonary embolism, acute cholecystitis, acute appendicitis, ectopic pregnancy, scrotal torsion, unexplained free intraperitoneal air, new or increasing intracranial hemorrhage, and malpositioned tubes and lines. The algorithms were developed using rule-based approaches and designed to search for common words and phrases in radiology reports that indicate critical results. Certain text-mining features were utilized such as wildcards, stemming, negation detection, proximity matching, and expanded searches with applicable synonyms. To further improve accuracy, the algorithms utilized modality and exam-specific queries, searched under the "Impression" field of the radiology report, and excluded reports with a low level of diagnostic certainty. Algorithm accuracy was determined using precision, recall, and F-measure using human review as the reference standard. The overall accuracy (F-measure) of the algorithms ranged from 81% to 100%, with a mean precision and recall of 96% and 91%, respectively. These algorithms can be applied to radiology report databases for quality assurance and accreditation, integrated with existing dashboards for display and monitoring, and ported to other institutions for their own use.

Automated detection of radiology reports that document non-routine communication of critical or significant results

The purpose of this investigation is to develop an automated method to accurately detect radiology reports that indicate non-routine communication of critical or significant results. Such a classification system would be valuable for performance monitoring and accreditation. Using a database of 2.3 million free-text radiology reports, a rule-based query algorithm was developed after analyzing hundreds of radiology reports that indicated communication of critical or significant results to a healthcare provider. This algorithm consisted of words and phrases used by radiologists to indicate such communications combined with specific handcrafted rules. This algorithm was iteratively refined and retested on hundreds of reports until the precision and recall did not significantly change between iterations. The algorithm was then validated on the entire database of 2.3 million reports, excluding those reports used during the testing and refinement process. Human review was used as the reference standard. The accuracy of this algorithm was determined using precision, recall, and F measure. Confidence intervals were calculated using the adjusted Wald method. The developed algorithm for detecting critical result communication has a precision of 97.0% (95% CI, 93.5–98.8%), recall 98.2% (95% CI, 93.4–100%), and F measure of 97.6% (ß = 1). Our query algorithm is accurate for identifying radiology reports that contain non-routine communication of critical or significant results. This algorithm can be applied to a radiology reports database for quality control purposes and help satisfy accreditation requirements.

Documentation of nonroutine communications of critical or significant radiology results: a multiyear experience at a tertiary hospital

PURPOSE

The aim of this study was to determine the frequency of radiology reports that contain nonroutine communications of results and categorize the urgency of such communications.

METHODS

A rule-based text-query algorithm was applied to a database of 2.3 million radiology reports, which has an accuracy of 98% for classifying reports containing documentation of communications. The frequency of such communications by year, modality, and study type was then determined. Finally, 200 random reports selected by the algorithm were analyzed, and reports containing critical results were categorized according to ascending levels of urgency.

RESULTS

Critical or noncritical results to health care providers were present in 5.09% of radiology reports (116,184 of 2,282,923). For common modalities, documentation of communications were most frequent in CT (14.34% [57,537 of 402,060]), followed by ultrasound (9.55% [17,814 of 186,626]), MRI (5.50% [13,697 of 248,833]), and chest radiography (1.57% [19,840 of 1,262,925]). From 1997 to 2005, there was an increase in reports containing such communications (3.04% in 1997, 6.82% in 2005). More reports contained nonroutine communications in single-view chest radiography (1.29% [5,533 of 428,377]) than frontal/lateral chest radiography (0.80% [1,815 of 226,837]), diagnostic mammography (9.42% [3,662 of 38,877]) than screening mammography (0.47% [289 of 61,114]), and head CT (26.21% [20,963 of 79,985]) than abdominal CT (15.05% [19,871 of 132,034]) or chest CT (5.33% [3,017 of 56,613]). All of these results were statistically significant (P < .00001). Of 200 random radiology reports indicating nonroutine communications, 155 (78%) had critical and 45 (22%) had noncritical results. Regarding level of urgency, 94 of 155 reports (60.6%) with critical results were categorized as high urgency, 31 (20.0%) as low urgency, 26 (16.8%) as medium urgency, and 4 (2.6%) as discrepant.

CONCLUSIONS

From 1997 to 2005, there was a significant increase in documentation of nonroutine communications, which may be due to increasing compliance with ACR guidelines. Most reports with nonroutine communications contain critical findings.

Informatics in radiology: an information model of the DICOM standard

The Digital Imaging and Communications in Medicine (DICOM) Standard is a key foundational technology for radiology. However, its complexity creates challenges for information system developers because the current DICOM specification requires human interpretation and is subject to nonstandard implementation. To address this problem, a formally sound and computationally accessible information model of the DICOM Standard was created. The DICOM Standard was modeled as an ontology, a machine-accessible and human-interpretable representation that may be viewed and manipulated by information-modeling tools. The DICOM Ontology includes a real-world model and a DICOM entity model. The real-world model describes patients, studies, images, and other features of medical imaging. The DICOM entity model describes connections between real-world entities and the classes that model the corresponding DICOM information entities. The DICOM Ontology was created to support the Cancer Biomedical Informatics Grid (caBIG) initiative, and it may be extended to encompass the entire DICOM Standard and serve as a foundation of medical imaging systems for research and patient care.