1
|
Friedman AB, Chen AT, Wu R, Coe NB, Halpern SD, Hwang U, Kelz RR, Cappola AR. Evaluation and disposition of older adults presenting to the emergency department with abdominal pain. J Am Geriatr Soc 2022; 70:501-511. [PMID: 34628638 PMCID: PMC10078825 DOI: 10.1111/jgs.17503] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 08/30/2021] [Accepted: 09/12/2021] [Indexed: 01/05/2023]
Abstract
BACKGROUND Abdominal pain is the most common chief complaint in US emergency departments (EDs) among patients over 65, who are at high risk of mortality or incident disability after the ED encounter. We sought to characterize the evaluation, management, and disposition of older adults who present to the ED with abdominal pain. METHODS We performed a survey-weighted analysis of the National Hospital Ambulatory Medical Care Survey (NHAMCS), comparing older adults with a chief complaint of abdominal pain to those without. Visits from 2013 to 2017 to nationally representative EDs were included. We analyzed 81,509 visits to 1211 US EDs, which projects to 531,780,629 ED visits after survey weighting. We report the diagnostic testing, evaluation, management, additional reasons for visit, and disposition of ED visits. RESULTS Among older adults (≥65 years), 7% of ED visits were for abdominal pain. Older patients with abdominal pain had a lower probability of being triaged to the "Emergent" (ESI2) acuity on arrival (7.1% vs. 14.8%) yet were more likely to be admitted directly to the operating room than older adults without abdominal pain (3.6% vs. 0.8%), with no statistically significant differences in discharge home, death, or admission to critical care. Ultrasound or CT imaging was performed in 60% of older adults with abdominal pain. A minority (39%) of older patients with abdominal pain received an electrocardiogram (EKG). CONCLUSIONS Abdominal pain in older adults presenting to EDs is a serious condition yet is triaged to "emergent" acuity at half the rate of other conditions. Opportunities for improving diagnosis and management may exist. Further research is needed to examine whether improved recognition of abdominal pain as a syndromic presentation would improve patient outcomes.
Collapse
Affiliation(s)
- Ari B. Friedman
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Angela T. Chen
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Rachel Wu
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Norma B. Coe
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Scott D. Halpern
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Ula Hwang
- Yale School of Medicine, Yale University, New Haven, Connecticut, USA
- Geriatrics Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, New York, USA
| | - Rachel R. Kelz
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Anne R. Cappola
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
2
|
Chillakuru YR, Munjal S, Laguna B, Chen TL, Chaudhari GR, Vu T, Seo Y, Narvid J, Sohn JH. Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing. BMC Med Inform Decis Mak 2021; 21:213. [PMID: 34253196 PMCID: PMC8276477 DOI: 10.1186/s12911-021-01574-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Accepted: 07/02/2021] [Indexed: 12/28/2022] Open
Abstract
Background A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text. Methods We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku. Results The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment. Conclusion We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.
Collapse
Affiliation(s)
- Yeshwant Reddy Chillakuru
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA.,The George Washington University School of Medicine and Health Sciences, 2300 I St NW, Washington, DC, 20052, USA
| | - Shourya Munjal
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA.,Rice University, 6100 Main St, Houston, TX, 77005, USA
| | - Benjamin Laguna
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Timothy L Chen
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Gunvant R Chaudhari
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Thienkhai Vu
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Youngho Seo
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Jared Narvid
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Jae Ho Sohn
- Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA.
| |
Collapse
|
3
|
Trivedi H, Mesterhazy J, Laguna B, Vu T, Sohn JH. Automatic Determination of the Need for Intravenous Contrast in Musculoskeletal MRI Examinations Using IBM Watson's Natural Language Processing Algorithm. J Digit Imaging 2019; 31:245-251. [PMID: 28924815 DOI: 10.1007/s10278-017-0021-3] [Citation(s) in RCA: 53] [Impact Index Per Article: 10.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Magnetic resonance imaging (MRI) protocoling can be time- and resource-intensive, and protocols can often be suboptimal dependent upon the expertise or preferences of the protocoling radiologist. Providing a best-practice recommendation for an MRI protocol has the potential to improve efficiency and decrease the likelihood of a suboptimal or erroneous study. The goal of this study was to develop and validate a machine learning-based natural language classifier that can automatically assign the use of intravenous contrast for musculoskeletal MRI protocols based upon the free-text clinical indication of the study, thereby improving efficiency of the protocoling radiologist and potentially decreasing errors. We utilized a deep learning-based natural language classification system from IBM Watson, a question-answering supercomputer that gained fame after challenging the best human players on Jeopardy! in 2011. We compared this solution to a series of traditional machine learning-based natural language processing techniques that utilize a term-document frequency matrix. Each classifier was trained with 1240 MRI protocols plus their respective clinical indications and validated with a test set of 280. Ground truth of contrast assignment was obtained from the clinical record. For evaluation of inter-reader agreement, a blinded second reader radiologist analyzed all cases and determined contrast assignment based on only the free-text clinical indication. In the test set, Watson demonstrated overall accuracy of 83.2% when compared to the original protocol. This was similar to the overall accuracy of 80.2% achieved by an ensemble of eight traditional machine learning algorithms based on a term-document matrix. When compared to the second reader's contrast assignment, Watson achieved 88.6% agreement. When evaluating only the subset of cases where the original protocol and second reader were concordant (n = 251), agreement climbed further to 90.0%. The classifier was relatively robust to spelling and grammatical errors, which were frequent. Implementation of this automated MR contrast determination system as a clinical decision support tool may save considerable time and effort of the radiologist while potentially decreasing error rates, and require no change in order entry or workflow.
Collapse
Affiliation(s)
- Hari Trivedi
- Radiology & Biomedical Imaging, UCSF Medical Center, 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Joseph Mesterhazy
- Radiology & Biomedical Imaging, UCSF Medical Center, 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Benjamin Laguna
- Radiology & Biomedical Imaging, UCSF Medical Center, 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Thienkhai Vu
- Radiology & Biomedical Imaging, UCSF Medical Center, 505 Parnassus Ave, San Francisco, CA, 94158, USA
| | - Jae Ho Sohn
- Radiology & Biomedical Imaging, UCSF Medical Center, 505 Parnassus Ave, San Francisco, CA, 94158, USA.
| |
Collapse
|
4
|
van Deen WK, Cho ES, Pustolski K, Wixon D, Lamb S, Valente TW, Menchine M. Involving end-users in the design of an audit and feedback intervention in the emergency department setting - a mixed methods study. BMC Health Serv Res 2019; 19:270. [PMID: 31035992 PMCID: PMC6489283 DOI: 10.1186/s12913-019-4084-3] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2018] [Accepted: 04/09/2019] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Long length of stays (LOS) in emergency departments (ED) negatively affect quality of care. Ordering of inappropriate diagnostic tests contributes to long LOS and reduces quality of care. One strategy to change practice patterns is to use performance feedback dashboards for physicians. While this strategy has proven to be successful in multiple settings, the most effective ways to deliver such interventions remain unknown. Involving end-users in the process is likely important for a successful design and implementation of a performance dashboard within a specific workplace culture. This mixed methods study aimed to develop design requirements for an ED performance dashboard and to understand the role of culture and social networks in the adoption process. METHODS We performed 13 semi-structured interviews with attending physicians in different roles within a single public ED in the U.S. to get an in-depth understanding of physicians' needs and concerns. Principles of human-centered design were used to translate these interviews into design requirements and to iteratively develop a front-end performance feedback dashboard. Pre- and post- surveys were used to evaluate the effect of the dashboard on physicians' motivation and to measure their perception of the usefulness of the dashboard. Data on the ED culture and underlying social network were collected. Outcomes were compared between physicians involved in the human-centered design process, those with exposure to the design process through the ED social network, and those with limited exposure. RESULTS Key design requirements obtained from the interviews were ease of access, drilldown functionality, customization, and a visual data display including monthly time-trends and blinded peer-comparisons. Identified barriers included concerns about unintended consequences and the veracity of underlying data. The surveys revealed that the ED culture and social network are associated with reported usefulness of the dashboard. Additionally, physicians' motivation was differentially affected by the dashboard based on their position in the social network. CONCLUSIONS This study demonstrates the feasibility of designing a performance feedback dashboard using a human-centered design approach in the ED setting. Additionally, we show preliminary evidence that the culture and underlying social network are of key importance for successful adoption of a dashboard.
Collapse
Affiliation(s)
- Welmoed K van Deen
- Gehr Family Center for Health Systems Science, Department of Medicine, Keck School of Medicine, University of Southern California, 2020 Zonal Ave, IRD 318, Los Angeles, CA, 90033, USA. .,Cedars-Sinai Center for Outcomes Research and Education, Department of Medicine, Division for Health Services Research, Cedars-Sinai Medical Center, 116 N. Robertson Boulevard, PACT 801, Los Angeles, CA, 90048, USA.
| | - Edward S Cho
- Keck School of Medicine, University of Southern California, 1975 Zonal Ave, Los Angeles, CA, 90033, USA
| | - Kathryn Pustolski
- Interactive Media & Games Division, School of Cinematic Arts, University of Southern California, 900 West 34th Street, Los Angeles, CA, 90089, USA
| | - Dennis Wixon
- Interactive Media & Games Division, School of Cinematic Arts, University of Southern California, 900 West 34th Street, Los Angeles, CA, 90089, USA
| | - Shona Lamb
- Keck School of Medicine, University of Southern California, 1975 Zonal Ave, Los Angeles, CA, 90033, USA
| | - Thomas W Valente
- Department of Preventive Medicine, Keck School of Medicine, University of Southern California, 2001 N Soto Street, Los Angeles, CA, 90032, USA
| | - Michael Menchine
- Department of Emergency Medicine, Keck School of Medicine, University of Southern California, 1200 N State Street, Room 1011, Los Angeles, CA, 90033, USA
| |
Collapse
|
5
|
Salerno S, Laghi A, Cantone MC, Sartori P, Pinto A, Frija G. Overdiagnosis and overimaging: an ethical issue for radiological protection. Radiol Med 2019; 124:714-720. [PMID: 30900132 DOI: 10.1007/s11547-019-01029-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2019] [Accepted: 03/11/2019] [Indexed: 02/06/2023]
Abstract
AIMS AND OBJECTIVES This study aimed to analyse the key factors that influence the overimaging using X-ray such as self-referral, defensive medicine and duplicate imaging studies and to emphasize the ethical problem that derives from it. MATERIALS AND METHODS In this study, we focused on the more frequent sources of overdiagnosis such as the total-body CT, proposed in the form of screening in both public and private sector, the choice of the most sensitive test for each pathology such as pulmonary embolism, ultrasound investigations mostly of the thyroid and of the prostate and MR examinations, especially of the musculoskeletal system. RESULTS The direct follow of overdiagnosis and overimaging is the increase in the risk of contrast media infusion, radiant damage, and costs in the worldwide healthcare system. The theme of the costs of overdiagnosis is strongly related to inappropriate or poorly appropriate imaging examination. CONCLUSIONS We underline the ethical imperatives of trust and right conduct, because the major ethical problems in radiology emerge in the justification of medical exposures of patients in the practice. A close cooperation and collaboration across all the physicians responsible for patient care in requiring imaging examination is also important, balancing possible ionizing radiation disadvantages and patient benefits in terms of care.
Collapse
Affiliation(s)
- Sergio Salerno
- Department of Diagnostic Radiology, University of Palermo, Policlinico Via del Vespro 127, 90127, Palermo, Italy.
| | - Andrea Laghi
- Department of Surgical and Medical Sciences and Translational Medicine, Sant'Andrea University Hospital, Sapienza-University of Rome, Via di Grottarossa 1035, 00189, Rome, Italy
| | - Marie-Claire Cantone
- Department of Biomedical, Surgical and Dental Sciences, University of Milan, Via Pascal 36, 20133, Milan, Italy
| | - Paolo Sartori
- Department of Radiology, SS Giovanni e Paolo Hospital, Castello 6777, 30122, Venice, Italy
| | - Antonio Pinto
- Department of Radiology, CTO Hospital, Azienda Ospedaliera dei Colli, Naples, Italy
| | - Guy Frija
- Department of Diagnostic Radiology, Hopital Européen Georges Pompidou Paris APHP, Université Paris-Descartes, Paris, France
| |
Collapse
|
6
|
Geist JR. The efficacy of diagnostic imaging should guide oral and maxillofacial radiology research. Oral Surg Oral Med Oral Pathol Oral Radiol 2017; 124:211-213. [PMID: 28698118 DOI: 10.1016/j.oooo.2017.06.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2017] [Accepted: 06/02/2017] [Indexed: 01/14/2023]
Affiliation(s)
- James R Geist
- Editor, OMR Section, Professor, Department of Biomedical and Diagnostic Sciences, University of Detroit Mercy School of Dentistry, Detroit, MI, USA
| |
Collapse
|
7
|
Brown AD, Marotta TR. A Natural Language Processing-based Model to Automate MRI Brain Protocol Selection and Prioritization. Acad Radiol 2017; 24:160-166. [PMID: 27889399 DOI: 10.1016/j.acra.2016.09.013] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2016] [Revised: 09/20/2016] [Accepted: 09/22/2016] [Indexed: 01/31/2023]
Abstract
RATIONALE AND OBJECTIVES Incorrect imaging protocol selection can contribute to increased healthcare cost and waste. To help healthcare providers improve the quality and safety of medical imaging services, we developed and evaluated three natural language processing (NLP) models to determine whether NLP techniques could be employed to aid in clinical decision support for protocoling and prioritization of magnetic resonance imaging (MRI) brain examinations. MATERIALS AND METHODS To test the feasibility of using an NLP model to support clinical decision making for MRI brain examinations, we designed three different medical imaging prediction tasks, each with a unique outcome: selecting an examination protocol, evaluating the need for contrast administration, and determining priority. We created three models for each prediction task, each using a different classification algorithm-random forest, support vector machine, or k-nearest neighbor-to predict outcomes based on the narrative clinical indications and demographic data associated with 13,982 MRI brain examinations performed from January 1, 2013 to June 30, 2015. Test datasets were used to calculate the accuracy, sensitivity and specificity, predictive values, and the area under the curve. RESULTS Our optimal results show an accuracy of 82.9%, 83.0%, and 88.2% for the protocol selection, contrast administration, and prioritization tasks, respectively, demonstrating that predictive algorithms can be used to aid in clinical decision support for examination protocoling. CONCLUSIONS NLP models developed from the narrative clinical information provided by referring clinicians and demographic data are feasible methods to predict the protocol and priority of MRI brain examinations.
Collapse
|
8
|
Marin JR, Mills AM. Developing a Research Agenda to Optimize Diagnostic Imaging in the Emergency Department: An Executive Summary of the 2015 Academic Emergency Medicine Consensus Conference. Acad Emerg Med 2015; 22:1363-71. [PMID: 26581181 DOI: 10.1111/acem.12818] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2015] [Accepted: 07/05/2015] [Indexed: 12/14/2022]
Abstract
The 2015 Academic Emergency Medicine (AEM) consensus conference, "Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization," was held on May 12, 2015, with the goal of developing a high-priority research agenda on which to base future research. The specific aims of the conference were to: 1) understand the current state of evidence regarding emergency department (ED) diagnostic imaging utilization and identify key opportunities, limitations, and gaps in knowledge; 2) develop a consensus-driven research agenda emphasizing priorities and opportunities for research in ED diagnostic imaging; and 3) explore specific funding mechanisms available to facilitate research in ED diagnostic imaging. Over a 2-year period, the executive committee and other experts in the field convened regularly to identify specific areas in need of future research. Six content areas within emergency diagnostic imaging were identified prior to the conference and served as the breakout groups on which consensus was achieved: clinical decision rules; use of administrative data; patient-centered outcomes research; training, education, and competency; knowledge translation and barriers to imaging optimization; and comparative effectiveness research in alternatives to traditional computed tomography use. The executive committee invited key stakeholders to assist with planning and to participate in the consensus conference to generate a multidisciplinary agenda. There were 164 individuals involved in the conference spanning various specialties, including emergency medicine (EM), radiology, surgery, medical physics, and the decision sciences. This issue of AEM is dedicated to the proceedings of the 16th annual AEM consensus conference as well as original research related to emergency diagnostic imaging.
Collapse
Affiliation(s)
- Jennifer R. Marin
- Departments of Pediatrics and Emergency Medicine; University of Pittsburgh School of Medicine; Pittsburgh PA
| | - Angela M. Mills
- Department of Emergency Medicine; Perelman School of Medicine at the University of Pennsylvania; Philadelphia PA
| |
Collapse
|