Urine testing is associated with inappropriate antibiotic use and increased length of stay in emergency department patients

Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models

BACKGROUND

Natural language processing (NLP) tools including recently developed large language models (LLMs) have myriad potential applications in medical care and research, including the efficient labeling and classification of unstructured text such as electronic health record (EHR) notes. This opens the door to large-scale projects that rely on variables that are not typically recorded in a structured form, such as patient signs and symptoms.

OBJECTIVES

This study is designed to acquaint the emergency medicine research community with the foundational elements of NLP, highlighting essential terminology, annotation methodologies, and the intricacies involved in training and evaluating NLP models. Symptom characterization is critical to urinary tract infection (UTI) diagnosis, but identification of symptoms from the EHR has historically been challenging, limiting large-scale research, public health surveillance, and EHR-based clinical decision support. We therefore developed and compared two NLP models to identify UTI symptoms from unstructured emergency department (ED) notes.

METHODS

The study population consisted of patients aged ≥ 18 who presented to an ED in a northeastern U.S. health system between June 2013 and August 2021 and had a urinalysis performed. We annotated a random subset of 1250 ED clinician notes from these visits for a list of 17 UTI symptoms. We then developed two task-specific LLMs to perform the task of named entity recognition: a convolutional neural network-based model (SpaCy) and a transformer-based model designed to process longer documents (Clinical Longformer). Models were trained on 1000 notes and tested on a holdout set of 250 notes. We compared model performance (precision, recall, F1 measure) at identifying the presence or absence of UTI symptoms at the note level.

RESULTS

A total of 8135 entities were identified in 1250 notes; 83.6% of notes included at least one entity. Overall F1 measure for note-level symptom identification weighted by entity frequency was 0.84 for the SpaCy model and 0.88 for the Longformer model. F1 measure for identifying presence or absence of any UTI symptom in a clinical note was 0.96 (232/250 correctly classified) for the SpaCy model and 0.98 (240/250 correctly classified) for the Longformer model.

CONCLUSIONS

The study demonstrated the utility of LLMs and transformer-based models in particular for extracting UTI symptoms from unstructured ED clinical notes; models were highly accurate for detecting the presence or absence of any UTI symptom on the note level, with variable performance for individual symptoms.

The importance of pharmacist engagement in diagnostic stewardship

Diagnostic stewardship is increasingly recognized as a powerful tool to improve patient safety. Given the close relationship between diagnostic testing and antimicrobial misuse, antimicrobial stewardship (AMS) pharmacists should be key members of the diagnostic team. Pharmacists practicing in AMS already frequently engage with clinicians to improve the diagnostic process and have many skills needed for the implementation of diagnostic stewardship initiatives. As diagnostic stewardship becomes more broadly used, all infectious disease clinicians, including pharmacists, must collaborate to optimize patient care.

Evaluation of the Diagnosis and Antibiotic Prescription Pattern in Patients Hospitalized with Urinary Tract Infections: Single-Center Study from a University-Affiliated Hospital

UTIs (urinary tract infections) are common bacterial infections with a non-negligible hospitalization rate. The diagnosis of UTIs remains a challenge for prescribers and a common source of misdiagnosis. This retrospective observational study aimed to evaluate whether recorded diagnosis by clinicians and empirical antibiotic therapy met the EAU (European Association of Urology) guideline in patients hospitalized with UTI. The study was conducted at an internal medicine unit of a tertiary care medical center in Hungary. The diagnosis was assessed based on clinical presentation, physical examination, and laboratory (including microbiological) results, considering all the potential risk factors. Diagnosis was considered misdiagnosis when not confirmed by clinical presentation or clinical signs and symptoms. Evaluation of empirical antibiotic therapy was performed only for confirmed UTIs. Empirical treatment was considered guideline-adherent when complying with the relevant recommendations. Out of 185 patients, 41.6% failed to meet EAU-based UTI diagnosis criteria, of which 27.6% were misdiagnosed and 14.1% were ABU (asymptomatic bacteriuria). The diagnosis of urosepsis recorded at admission (9.7%, 18/185) was not confirmed either by clinical or microbiological tests in five (5/18) cases. The initial empirical therapies for UTI showed a relatively low rate (45.4%) of guideline adherence regarding agent selection. The most common guideline-non-adherent therapies were combinations with metronidazole (16.7%). Dosage appropriateness assessments showed a guideline adherence rate of 36.1%, and underdosing due to high body weight was common (9.3%). Overall (agent, route of administration, dose, duration) guideline adherence was found to be substantially low (10.2%). We found a relatively high rate of misdiagnosed UTIs. Written protocols on the ward may be crucial in reducing misdiagnosis and in optimizing antibiotic use.

Improving management of hyponatraemia by increasing urine testing in the emergency department

Hyponatraemia on hospital admission is associated with increased length of stay, healthcare expenditures and mortality. Urine studies collected before fluid or diuretic administration are essential to diagnose the underlying cause of hyponatraemia, thereby empowering admitting teams to employ the appropriate treatment. A multidisciplinary quality improvement (QI) team led by internal medicine residents performed a QI project from July 2020 through June 2021 to increase the rate of urine studies collected before fluid or diuretic administration in the emergency department (ED) in patients admitted with moderate to severe hyponatraemia. We implemented two plan-do-study-act (PDSA) cycles to address this goal. In PDSA Cycle #1, we displayed an educational poster in employee areas of the ED and met with nursing staff at their monthly meetings to communicate the project and answer questions. We also obtained agreement from ED attending physicians and nursing leaders to support the project. In PDSA Cycle #2, we implemented a structural change in the nursing triage process to issue every patient who qualified for bloodwork with a urine specimen container labelled with a medical record number on registration so that the patient could provide a sample at any point, including while in the waiting area. After PDSA Cycle #1, urine specimen collection increased from 34.5% to 57.5%. After PDSA Cycle #2, this increased further to 59%. We conclude that a combination of educational and structural changes led to a significant increase in urine specimen collection before fluid or diuretic administration among patients presenting with moderate-to-severe hyponatraemia in the ED.