Improving the Quality of Imaging in the Emergency Department

Evaluation and disposition of older adults presenting to the emergency department with abdominal pain

BACKGROUND

Abdominal pain is the most common chief complaint in US emergency departments (EDs) among patients over 65, who are at high risk of mortality or incident disability after the ED encounter. We sought to characterize the evaluation, management, and disposition of older adults who present to the ED with abdominal pain.

METHODS

We performed a survey-weighted analysis of the National Hospital Ambulatory Medical Care Survey (NHAMCS), comparing older adults with a chief complaint of abdominal pain to those without. Visits from 2013 to 2017 to nationally representative EDs were included. We analyzed 81,509 visits to 1211 US EDs, which projects to 531,780,629 ED visits after survey weighting. We report the diagnostic testing, evaluation, management, additional reasons for visit, and disposition of ED visits.

RESULTS

Among older adults (≥65 years), 7% of ED visits were for abdominal pain. Older patients with abdominal pain had a lower probability of being triaged to the "Emergent" (ESI2) acuity on arrival (7.1% vs. 14.8%) yet were more likely to be admitted directly to the operating room than older adults without abdominal pain (3.6% vs. 0.8%), with no statistically significant differences in discharge home, death, or admission to critical care. Ultrasound or CT imaging was performed in 60% of older adults with abdominal pain. A minority (39%) of older patients with abdominal pain received an electrocardiogram (EKG).

CONCLUSIONS

Abdominal pain in older adults presenting to EDs is a serious condition yet is triaged to "emergent" acuity at half the rate of other conditions. Opportunities for improving diagnosis and management may exist. Further research is needed to examine whether improved recognition of abdominal pain as a syndromic presentation would improve patient outcomes.

Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing

Background

A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text.

Methods

We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku.

Results

The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment.

Conclusion

We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.

Automatic Determination of the Need for Intravenous Contrast in Musculoskeletal MRI Examinations Using IBM Watson's Natural Language Processing Algorithm

Magnetic resonance imaging (MRI) protocoling can be time- and resource-intensive, and protocols can often be suboptimal dependent upon the expertise or preferences of the protocoling radiologist. Providing a best-practice recommendation for an MRI protocol has the potential to improve efficiency and decrease the likelihood of a suboptimal or erroneous study. The goal of this study was to develop and validate a machine learning-based natural language classifier that can automatically assign the use of intravenous contrast for musculoskeletal MRI protocols based upon the free-text clinical indication of the study, thereby improving efficiency of the protocoling radiologist and potentially decreasing errors. We utilized a deep learning-based natural language classification system from IBM Watson, a question-answering supercomputer that gained fame after challenging the best human players on Jeopardy! in 2011. We compared this solution to a series of traditional machine learning-based natural language processing techniques that utilize a term-document frequency matrix. Each classifier was trained with 1240 MRI protocols plus their respective clinical indications and validated with a test set of 280. Ground truth of contrast assignment was obtained from the clinical record. For evaluation of inter-reader agreement, a blinded second reader radiologist analyzed all cases and determined contrast assignment based on only the free-text clinical indication. In the test set, Watson demonstrated overall accuracy of 83.2% when compared to the original protocol. This was similar to the overall accuracy of 80.2% achieved by an ensemble of eight traditional machine learning algorithms based on a term-document matrix. When compared to the second reader's contrast assignment, Watson achieved 88.6% agreement. When evaluating only the subset of cases where the original protocol and second reader were concordant (n = 251), agreement climbed further to 90.0%. The classifier was relatively robust to spelling and grammatical errors, which were frequent. Implementation of this automated MR contrast determination system as a clinical decision support tool may save considerable time and effort of the radiologist while potentially decreasing error rates, and require no change in order entry or workflow.

Involving end-users in the design of an audit and feedback intervention in the emergency department setting - a mixed methods study

BACKGROUND

Long length of stays (LOS) in emergency departments (ED) negatively affect quality of care. Ordering of inappropriate diagnostic tests contributes to long LOS and reduces quality of care. One strategy to change practice patterns is to use performance feedback dashboards for physicians. While this strategy has proven to be successful in multiple settings, the most effective ways to deliver such interventions remain unknown. Involving end-users in the process is likely important for a successful design and implementation of a performance dashboard within a specific workplace culture. This mixed methods study aimed to develop design requirements for an ED performance dashboard and to understand the role of culture and social networks in the adoption process.

METHODS

We performed 13 semi-structured interviews with attending physicians in different roles within a single public ED in the U.S. to get an in-depth understanding of physicians' needs and concerns. Principles of human-centered design were used to translate these interviews into design requirements and to iteratively develop a front-end performance feedback dashboard. Pre- and post- surveys were used to evaluate the effect of the dashboard on physicians' motivation and to measure their perception of the usefulness of the dashboard. Data on the ED culture and underlying social network were collected. Outcomes were compared between physicians involved in the human-centered design process, those with exposure to the design process through the ED social network, and those with limited exposure.

RESULTS

Key design requirements obtained from the interviews were ease of access, drilldown functionality, customization, and a visual data display including monthly time-trends and blinded peer-comparisons. Identified barriers included concerns about unintended consequences and the veracity of underlying data. The surveys revealed that the ED culture and social network are associated with reported usefulness of the dashboard. Additionally, physicians' motivation was differentially affected by the dashboard based on their position in the social network.

CONCLUSIONS

This study demonstrates the feasibility of designing a performance feedback dashboard using a human-centered design approach in the ED setting. Additionally, we show preliminary evidence that the culture and underlying social network are of key importance for successful adoption of a dashboard.

Overdiagnosis and overimaging: an ethical issue for radiological protection

AIMS AND OBJECTIVES

This study aimed to analyse the key factors that influence the overimaging using X-ray such as self-referral, defensive medicine and duplicate imaging studies and to emphasize the ethical problem that derives from it.

MATERIALS AND METHODS

In this study, we focused on the more frequent sources of overdiagnosis such as the total-body CT, proposed in the form of screening in both public and private sector, the choice of the most sensitive test for each pathology such as pulmonary embolism, ultrasound investigations mostly of the thyroid and of the prostate and MR examinations, especially of the musculoskeletal system.

RESULTS

The direct follow of overdiagnosis and overimaging is the increase in the risk of contrast media infusion, radiant damage, and costs in the worldwide healthcare system. The theme of the costs of overdiagnosis is strongly related to inappropriate or poorly appropriate imaging examination.

CONCLUSIONS

We underline the ethical imperatives of trust and right conduct, because the major ethical problems in radiology emerge in the justification of medical exposures of patients in the practice. A close cooperation and collaboration across all the physicians responsible for patient care in requiring imaging examination is also important, balancing possible ionizing radiation disadvantages and patient benefits in terms of care.

A Natural Language Processing-based Model to Automate MRI Brain Protocol Selection and Prioritization

RATIONALE AND OBJECTIVES

Incorrect imaging protocol selection can contribute to increased healthcare cost and waste. To help healthcare providers improve the quality and safety of medical imaging services, we developed and evaluated three natural language processing (NLP) models to determine whether NLP techniques could be employed to aid in clinical decision support for protocoling and prioritization of magnetic resonance imaging (MRI) brain examinations.

MATERIALS AND METHODS

To test the feasibility of using an NLP model to support clinical decision making for MRI brain examinations, we designed three different medical imaging prediction tasks, each with a unique outcome: selecting an examination protocol, evaluating the need for contrast administration, and determining priority. We created three models for each prediction task, each using a different classification algorithm-random forest, support vector machine, or k-nearest neighbor-to predict outcomes based on the narrative clinical indications and demographic data associated with 13,982 MRI brain examinations performed from January 1, 2013 to June 30, 2015. Test datasets were used to calculate the accuracy, sensitivity and specificity, predictive values, and the area under the curve.

RESULTS

Our optimal results show an accuracy of 82.9%, 83.0%, and 88.2% for the protocol selection, contrast administration, and prioritization tasks, respectively, demonstrating that predictive algorithms can be used to aid in clinical decision support for examination protocoling.

CONCLUSIONS

NLP models developed from the narrative clinical information provided by referring clinicians and demographic data are feasible methods to predict the protocol and priority of MRI brain examinations.

Developing a Research Agenda to Optimize Diagnostic Imaging in the Emergency Department: An Executive Summary of the 2015 Academic Emergency Medicine Consensus Conference

The 2015 Academic Emergency Medicine (AEM) consensus conference, "Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization," was held on May 12, 2015, with the goal of developing a high-priority research agenda on which to base future research. The specific aims of the conference were to: 1) understand the current state of evidence regarding emergency department (ED) diagnostic imaging utilization and identify key opportunities, limitations, and gaps in knowledge; 2) develop a consensus-driven research agenda emphasizing priorities and opportunities for research in ED diagnostic imaging; and 3) explore specific funding mechanisms available to facilitate research in ED diagnostic imaging. Over a 2-year period, the executive committee and other experts in the field convened regularly to identify specific areas in need of future research. Six content areas within emergency diagnostic imaging were identified prior to the conference and served as the breakout groups on which consensus was achieved: clinical decision rules; use of administrative data; patient-centered outcomes research; training, education, and competency; knowledge translation and barriers to imaging optimization; and comparative effectiveness research in alternatives to traditional computed tomography use. The executive committee invited key stakeholders to assist with planning and to participate in the consensus conference to generate a multidisciplinary agenda. There were 164 individuals involved in the conference spanning various specialties, including emergency medicine (EM), radiology, surgery, medical physics, and the decision sciences. This issue of AEM is dedicated to the proceedings of the 16th annual AEM consensus conference as well as original research related to emergency diagnostic imaging.