Building large-scale registries from unstructured clinical notes using a low-resource natural language processing pipeline

Building clinical registries is an important step in clinical research and improvement of patient care quality. Natural Language Processing (NLP) methods have shown promising results in extracting valuable information from unstructured clinical notes. However, the structure and nature of clinical notes are very different from regular text that state-of-the-art NLP models are trained and tested on, and they have their own set of challenges. In this study, we propose Sentence Extractor with Keywords (SE-K), an efficient and interpretable classification approach for extracting information from clinical notes and show that it outperforms more computationally expensive methods in text classification. Following the Institutional Review Board (IRB) approval, we used SE-K and two embedding based NLP approaches (Sentence Extractor with Embeddings (SE-E) and Bidirectional Encoder Representations from Transformers (BERT)) to develop comprehensive registry of anterior cruciate ligament surgeries from 20 years of unstructured clinical data at a multi-site tertiary-care regional children's hospital. The low-resource approach (SE-K) had better performance (average AUROC of 0.94 ± 0.04) than the embedding-based approaches (SE-E: 0.93 ± 0.04 and BERT: 0.87 ± 0.09) for out of sample validation, in addition to minimum performance drop between test and out-of-sample validation. Moreover, the SE-K approach was at least six times faster (on CPU) than SE-E (on CPU) and BERT (on GPU) and provides interpretability. Our proposed approach, SE-K, can be effectively used to extract relevant variables from clinic notes to build large-scale registries, with consistently better performance compared to the more resource-intensive approaches (e.g., BERT). Such approaches can facilitate information extraction from unstructured notes for registry building, quality improvement and adverse event monitoring.

[Patient-reported outcomes in chronic diseases under treatment with cannabis medicines : Analysis of the results of the Copeia survey]

BACKGROUND

The survey of Copeia captured early 2022 patient-reported outcomes (PRO) in Germany under cannabis medicinal product (CAM) therapy, with particular attention to symptoms, symptom changes, indications, side effects, dosages, and cost bearers.

GOAL

This study investigated the question of whether associations emerge from the results that could play a role in the indication and treatment monitoring of CAM in chronically ill patients.

MATERIALS AND METHODS

A standardized questionnaire was administered online nationwide in dialogue form over a 15-week period to collect itemized symptoms and PRO. Recruitment was supported by pharmacies, prescribing physicians, and patient associations. Inclusion criteria included physician-prescribed CAM therapy.

RESULTS AND DISCUSSION

Of 1582 participants, 1030 data sets (65%) could be completely analyzed. There was a heterogeneous patient population, whose common feature was disease chronicity. The frequency distribution of symptoms showed a homogeneous pattern for the respective indications, in which the most frequent six (pain 71%, sleep disturbance 64%, stress/tension 52%, inner restlessness 52%, depressive mood 44% and muscle tension 43%) seem to have a special significance. According to subjective assessment, quality of life improved significantly in 84% of all participating patients.

CONCLUSION

A symptom matrix (SMX) composed of different symptoms seems to play a special role in CAM therapy to improve the quality of life of chronically ill patients, regardless of the underlying disease. The SMX could contribute to the identification of an indication and to targeted treatment monitoring.

Enhancing Pressure Injury Surveillance Using Natural Language Processing

OBJECTIVE

This study assessed the feasibility of nursing handoff notes to identify underreported hospital-acquired pressure injury (HAPI) events.

METHODS

We have established a natural language processing-assisted manual review process and workflow for data extraction from a corpus of nursing notes across all medical inpatient and intensive care units in a tertiary care pediatric center. This system is trained by 2 domain experts. Our workflow started with keywords around HAPI and treatments, then regular expressions, distributive semantics, and finally a document classifier. We generated 3 models: a tri-gram classifier, binary logistic regression model using the regular expressions as predictors, and a random forest model using both models together. Our final output presented to the event screener was generated using a random forest model validated using derivation and validation sets.

RESULTS

Our initial corpus involved 70,981 notes during a 1-year period from 5484 unique admissions for 4220 patients. Our interrater human reviewer agreement on identifying HAPI was high ( κ = 0.67; 95% confidence interval [CI], 0.58-0.75). Our random forest model had 95% sensitivity (95% CI, 90.6%-99.3%), 71.2% specificity (95% CI, 65.1%-77.2%), and 78.7% accuracy (95% CI, 74.1%-83.2%). A total of 264 notes from 148 unique admissions (2.7% of all admissions) were identified describing likely HAPI. Sixty-one described new injuries, and 64 describe known yet possibly evolving injuries. Relative to the total patient population during our study period, HAPI incidence was 11.9 per 1000 discharges, and incidence rate was 1.2 per 1000 bed-days.

CONCLUSIONS

Natural language processing-based surveillance is proven to be feasible and high yield using nursing handoff notes.

Natural language processing for identification of refractory status epilepticus in children

OBJECTIVE

Pediatric status epilepticus is one of the most frequent pediatric emergencies, with high mortality and morbidity. Utilizing electronic health records (EHRs) permits analysis of care approaches and disease outcomes at a lower cost than prospective research. However, reviewing EHR manually is time intensive. We aimed to compare refractory status epilepticus (rSE) cases identified by human EHR review with a natural language processing (NLP)-assisted rSE screen followed by a manual review.

METHODS

We used the NLP screening tool Document Review Tool (DrT) to generate regular expressions, trained a bag-of-words NLP classifier on EHRs from 2017 to 2019, and then tested our algorithm on data from February to December 2012. We compared results from manual review to NLP-assisted search followed by manual review.

RESULTS

Our algorithm identified 1528 notes in the test set. After removing notes pertaining to the same event by DrT, the user reviewed a total number of 400 notes to find patients with rSE. Within these 400 notes, we identified 31 rSE cases, including 12 new cases not found in manual review, and 19 of the 20 previously identified cases. The NLP-assisted model found 31 of 32 cases, with a sensitivity of 96.88% (95% CI = 82%-99.84%), whereas manual review identified 20 of 32 cases, with a sensitivity of 62.5% (95% CI = 43.75%-78.34%).

SIGNIFICANCE

DrT provided a highly sensitive model compared to human review and an increase in patient identification through EHRs. The use of DrT is a suitable application of NLP for identifying patients with a history of recent rSE, which ultimately contributes to the implementation of monitoring techniques and treatments in near real time.

Bacteremia in Children With Fever and Acute Lower Extremity Pain

BACKGROUND AND OBJECTIVES

Blood cultures (BCxs) are often obtained in the initial evaluation of children with fever and acute lower extremity pain; however, their yield in this population is unknown. We aim to describe the prevalence of bacteremia among children presenting to the emergency department (ED) with fever and acute lower extremity pain and identify predictors of bacteremia.

METHODS

Cross-sectional review of children aged 1 to 18 years presenting to the ED with fever and acute lower extremity pain between 2010 and 2020. We excluded patients with trauma within the previous 24 hours, orthopedic comorbidity, immunocompromised status, or antibiotic pretreatment. We identified our cohort using a Natural Language Processing-assisted model with manual review and abstracted clinical data. Our primary outcome was a BCx positive for a pathogen.

RESULTS

We screened 478 979 ED notes and identified 689 patients who met inclusion criteria. Median age was 5.3 years (interquartile range 2.7-8.8); 39.5% were female. BCxs were obtained from 75.9% (523/689) of patients, of which 510 were available for review. BCxs were positive in 70/510 (13.7%; 95% CI, confidence interval [CI], 10.9-17.0) of children and in 70/689 (10.2%; 95% CI, 8.0-12.7%) of the entire cohort. The most common pathogens were methicillin-susceptible Staphylococcus aureus (71.6%) and methicillin-resistant Staphylococcus aureus (15.7%). Predictors of bacteremia include C-reactive protein ≥3 mg/dL (odds ratio, 4.5; 95% CI, 2.1-9.6) and localizing examination findings (odds ratio, 3.3; 95% CI, 1.4-7.9).

CONCLUSIONS

The prevalence of bacteremia among children presenting to the ED with fever and acute lower extremity pain is high. Routine BCx should be considered in the initial evaluation of this population.

Electronic surveillance of patient safety events using natural language processing

Objective

We describe our approach to surveillance of reportable safety events captured in hospital data including free-text clinical notes. We hypothesize that a) some patient safety events are documented only in the clinical notes and not in any other accessible source; and b) large-scale abstraction of event data from clinical notes is feasible.

Materials and Methods

We use regular expressions to generate a training data set for a machine learning model and apply this model to the full set of clinical notes and conduct further review to identify safety events of interest. We demonstrate this approach on peripheral intravenous (PIV) infiltrations and extravasations (PIVIEs).

Results

During Phase 1, we collected 21,362 clinical notes, of which 2342 were reviewed. We identified 125 PIV events, of which 44 cases (35%) were not captured by other patient safety systems. During Phase 2, we collected 60,735 clinical notes and identified 440 infiltrate events. Our classifier demonstrated accuracy above 90%.

Conclusion

Our method to identify safety events from the free text of clinical documentation offers a feasible and scalable approach to enhance existing patient safety systems. Expert reviewers, using a machine learning model, can conduct routine surveillance of patient safety events.

Utility of Blood Cultures and Empiric Antibiotics in Febrile Pediatric Hemophilia Patients With Central Venous Access Devices

BACKGROUND

Children with hemophilia frequently require long-term central venous access devices (CVADs) for regular infusion of factor products. Hemophilia patients are not immunocompromised, but the presence and use of CVADs are associated with infections including bacteremia. Currently, the utility of blood cultures in evaluation of the febrile hemophilia patient with an indwelling CVAD is unknown, nor is optimal empiric antibiotic use.

METHODS

We performed a retrospective cross-sectional study of febrile immunocompetent hemophilia patients with CVADs presenting to a large academic urban pediatric emergency department from 1995 to 2017. We used a natural language processing electronic search, followed by manual chart review to construct the cohort. We analyzed rate of pathogen recovery from cultures of blood in subgroups of hemophilia patients, the pathogen profile, and the reported pathogen susceptibilities to ceftriaxone.

RESULTS

Natural language processing electronic search identified 181 visits for fever among hemophilia patients with indwelling CVADs of which 147 cases from 44 unique patients met study criteria. Cultures of blood were positive in 56 (38%) of 147 patients (95% confidence interval, 30%-47%). Seventeen different organisms were isolated (10 pathogens and 7 possible pathogens) with Staphylococcus aureus and coagulase-negative Staphylococcus species as the most common. Thirty-four percent of isolates were reported as susceptible to ceftriaxone. Positive blood cultures were more common in cases involving patients with inhibitors (n = 71) versus those without (n = 76), odds ratio, 7.4 (95% confidence interval, 3.5-15.9). This was observed irrespective of hemophilia type.

CONCLUSIONS

Febrile immunocompetent hemophilia patients with indwelling CVADs have high rates of bacteremia. Empiric antimicrobial therapy should be targeted to anticipated pathogens and take into consideration local susceptibility patterns for Staphylococcus aureus.

Is lymphangitic streaking associated with different pathogens?

OBJECTIVES

Little is known regarding the differences in microbiology associated with cellulitis or abscess with or without lymphangitic streaking. The objective of our study is to assess whether there are differences in the pathogens identified from wound cultures of patients with paronychia with and without associated lymphangitis.

METHODS

Retrospective cross-sectional study at a tertiary pediatric emergency department over 25 years. We opted to assess patients with paronychia of the finger, assuming that these cases will have a greater variety of causative pathogens compared to other cases of cellulitis and soft tissue abscess that are associated with nail biting. Case identification was conducted using a computerized text-screening search that was refined by manual chart review. We included patients from 1 month to 20 years of age who underwent an incision and drainage (I&D) of a paronychia and had a culture obtained. The presence or absence of lymphangitis was determined from the clinical narrative in the medical record. We excluded patients treated with antibiotics prior to I&D as well as immune-compromised patients. We used descriptive statistics for prevalence and χ2 tests for categorical variables.

RESULTS

Two hundred sixty-six patients met inclusion criteria. The median age was 9.7 years [IQR 4.7, 15.4] and 45.1% were female. Twenty-two patients (8.3%) had lymphangitic streaking associated with their paronychia. Patients with lymphangitis streaking were similar to those without lymphangitis in terms of age and sex (p = 0.52 and p = 0.82, respectively). Overall, the predominant bacteria was MSSA (40%) followed by MRSA (26%). No significant differences were found between the pathogens in the 22 patients with associated lymphangitis compared to the 244 patients without.

CONCLUSION

Staphylococcus aureus represent the majority of pathogens in paronychia, although streptococcal species and gram-negative bacteria were also common. Among patients with paronychia of the finger, there seems to be no association between pathogen type and presence of lymphangitic streaking.

Predictors of Primary Intracranial Hypertension in Children Using a Newly Suggested Opening Pressure Cutoff of 280 mm H2O

OBJECTIVES

We assessed the clinical characteristics of primary intracranial hypertension (PIH) in children using a newly recommended threshold for cerebrospinal fluid opening pressure (280 mm H₂O).

METHOD

Cross-sectional study of patients age ≤21 years who had a lumbar puncture done for evaluation of PIH. Patients were excluded if lumbar puncture was done for a suspected infection, seizure, mental status changes, multiple sclerosis, or Guillain-Barre syndrome. Cases were identified using a text-search module followed by manual review. We performed χ2 analysis for categorical data and Mann-Whitney U test for continuous data, followed by a binary logistic regression.

RESULTS

We identified 374 patients of whom 67% were female, median age was 13 years interquartile range (11 to 16 years), and admission rate was 24%. Using an opening pressure cutoff of 250 mm H₂O, 127 patients (34%) were identified as having PIH, whereas using the new cutoff 105 patients (28%) met PIH criteria. Predictors for PIH included optic disc edema or sixth nerve palsy using both old, odds ratio (OR) 7.6 (4.3, 13.5), and new cutoffs, OR 9.7 (95% confidence interval 5.1, 18.5). Headache duration ≤61 days is predictive of PIH using the new cutoff OR 4.1 (95% confidence interval 1.3, 12.8). A model is presented which stratifies patients into groups with low (7%), medium (18%), and high (greater than 42%) risk of PIH.

CONCLUSIONS

A higher cerebrospinal fluid opening pressure threshold in the criteria of PIH is associated with PIH patients with a different symptom profile. Children with optic disc edema, bulging fontanel or sixth nerve palsy, are at increased risk for PIH.

Pediatric Emergency Department Visits for Homelessness After Shelter Eligibility Policy Change

BACKGROUND AND OBJECTIVES

In 2012, Massachusetts changed its emergency shelter eligibility policy for homeless families. One new criterion to document homelessness was staying in a location "not meant for human habitation," and the emergency department (ED) fulfilled this requirement. Our aim for this study is to analyze the frequency and costs of pediatric ED visits for homelessness before and after this policy.

METHODS

This is a retrospective study of ED visits for homelessness at a children's hospital from March 2010 to February 2016. A natural language processing tool was used to identify cases, which were manually reviewed for inclusion. We compared demographic and homelessness circumstance characteristics and conducted an interrupted time series analysis to compare ED visits by homeless children before and after the policy. We compared the change in ED visits for homelessness to the number of homeless children in Massachusetts. We analyzed payment data for each visit.

RESULTS

There were 312 ED visits for homelessness; 95% (n = 297) of visits were after the policy. These visits increased 4.5 times after the policy (95% confidence interval: 1.33 to 15.23). Children seen after the policy were more likely to have no medical complaint (rate ratio: 3.27; 95% confidence interval: 1.18 to 9.01). Although the number of homeless children in Massachusetts increased 1.4 times over the study period, ED visits for homelessness increased 13-fold. Payments (average: $557 per visit) were >4 times what a night in a shelter would cost; 89% of payments were made through state-based insurance plans.

CONCLUSIONS

A policy change to Massachusetts' shelter eligibility was associated with increased pediatric ED visits for homelessness along with substantial health care costs.

A method to identify pediatric high-risk diagnoses missed in the emergency department

BACKGROUND

Diagnostic error can lead to increased morbidity, mortality, healthcare utilization and cost. The 2015 National Academy of Medicine report "Improving Diagnosis in Healthcare" called for improving diagnostic accuracy by developing innovative electronic approaches to reduce medical errors, including missed or delayed diagnosis. The objective of this article was to develop a process to detect potential diagnostic discrepancy between pediatric emergency and inpatient discharge diagnosis using a computer-based tool facilitating expert review.

METHODS

Using a literature search and expert opinion, we identified 10 pediatric diagnoses with potential for serious consequences if missed or delayed. We then developed and applied a computerized tool to identify linked emergency department (ED) encounters and hospitalizations with these discharge diagnoses. The tool identified discordance between ED and hospital discharge diagnoses. Cases identified as discordant were manually reviewed by pediatric emergency medicine experts to confirm discordance.

RESULTS

Our computerized tool identified 55,233 ED encounters for hospitalized children over a 5-year period, of which 2161 (3.9%) had one of the 10 selected high-risk diagnoses. After expert record review, we identified 67 (3.1%) cases with discordance between ED and hospital discharge diagnoses. The most common discordant diagnoses were Kawasaki disease and pancreatitis.

CONCLUSIONS

We successfully developed and applied a semi-automated process to screen a large volume of hospital encounters to identify discordant diagnoses for selected pediatric medical conditions. This process may be valuable for informing and improving ED diagnostic accuracy.

Predictors of a drainable suppurative adenitis among children presenting with cervical adenopathy

OBJECTIVES

We sought to identify predictors for a drainable suppurative adenitis [DSA] among patients presenting with acute cervical lymphadenitis.

METHODS

A retrospective cross sectional study of all patients admitted to an urban pediatric tertiary care emergency department over a 15 year period. Otherwise healthy patients who underwent imaging for an evaluation of cervical lymphadenitis were included. Cases were identified using a text-search module followed by manual review. We excluded immunocompromised patients and those with lymphadenopathy felt to be not directly infected (i.e. reactive) or that was not acute (symptom duration >28 days). Data collected included: age, gender, duration of symptoms, highest recorded temperature, physical exam findings, laboratory and imaging results, and surgical findings. A DSA was defined as >1.5 cm in diameter on imaging. We performed binary logistic regression to determine independent clinical predictors of a DSA.

RESULTS

Three hundred sixty-one patients met inclusion criteria. Three hundred six patients (85%) had a CT scan, 55 (15%) had an ultrasound and 33 (9%) had both. DSA was identified in 71 (20%) patients. Clinical features independently associated with a DSA included absence of clinical pharyngitis, WBC >15,000/mm³, age ≤3 years, anterior cervical chain location, largest palpable diameter on exam >3 cm and prior antibiotic treatment of >24 h. The presence of fever, skin erythema, or fluctuance on examination, was not found to be predictive of DSA.

CONCLUSIONS

We identified independent predictors of DSA among children presenting with cervical adenitis. Risk can be stratified into risk groups based on these clinical features.

Diagnostic Lumbar Puncture Among Children With Facial Palsy in a Lyme Disease Endemic Area

We identified 620 children with peripheral facial palsy of which 211 (34%) had Lyme disease. The 140 children who had a lumbar puncture performed were more likely to be hospitalized (73% LP performed vs 2% no LP) and to receive parenteral antibiotics (62% LP performed vs 6% no LP).

How Much Cerebrospinal Fluid Should We Remove Prior to Measuring a Closing Pressure?

OBJECTIVE

The objective of this study was to identify a relationship between cerebrospinal fluid (CSF) volume removal and change in CSF pressure in children with suspected idiopathic intracranial hypertension (IIH).

METHODS

We performed a cross-sectional study of children 22 years and younger who underwent a lumbar puncture (LP) and had a documented opening pressure, closing pressure, and volume removed. Relationship between volume removal and pressure change was determined using a fractional polynomial regression procedure.

RESULTS

In the 297 patients who met the inclusion criteria, CSF pressure decreased by 1 cm H₂O for every 0.91 mL of CSF removed if the maximum change in pressure was less than 15 cm H₂O ( R² = 0.38).

CONCLUSION

A linear relationship exists between the volume of CSF removed and the amount of pressure relieved when the desired pressure change is less than 15 cm H₂O.

The Yield of Neuroimaging in Children Presenting to the Emergency Department With Acute Ataxia in the Post-Varicella Vaccine Era

To determine the yield of neuroimaging in children presenting to the emergency department with acute ataxia in the post-varicella vaccine era, we conducted a cross-sectional study between 1995 and 2013 at a single pediatric tertiary care center. We included children aged 1-18 years evaluated for acute ataxia of <7 days' duration. The main outcome was clinically urgent intracranial pathology defined as a radiologic finding that changed initial management. We identified 364 children, among whom neuroimaging was obtained in 284 (78%). Forty-two children had clinically urgent intracranial pathology (13%, 95% confidence interval 9%-17%); tumors and acute disseminated encephalomyelitis were the leading findings. Age ≤3 years and symptoms ≤3 days of duration were predictors of low risk (0.7%, 95% confidence interval 0%-4.4%). In conclusion, neuroimaging may be indicated for most patients presenting with acute ataxia. Neuroimaging may be deferred in younger children with short duration of symptoms contingent on close follow-up.

The Frequency of Postreduction Interventions After Successful Enema Reduction of Intussusception

OBJECTIVES

The objective was to determine the frequency of postreduction, hospital-level interventions among children with successful reduction of ileocolic intussusception and identify factors that predict the need for such interventions.

METHODS

This was a retrospective cross-sectional study of children who underwent successful enema reduction for ileocolic intussusception at a single emergency department. Hospital-level interventions were included if they occurred within 24 hours of reduction and were further classified as either major (recurrence or possible perforation) or minor (imaging for suspected recurrence or administration of parenteral narcotics or antiemetics). Binary logistic regression was used to identify predictors for hospital-level interventions.

RESULTS

A total of 464 children underwent enema reduction. The median age was 1.7 years (interquartile range [IQR] = 0.8 to 2.5 years), and 66% were male. A total of 435 (94%) were hospitalized with a median hospital stay of 25 hours (IQR = 19 to 34 hours). Nineteen percent (95% confidence interval [CI] = 15% to 22%) needed postreduction interventions, including 6% (95% CI = 4% to 9%) who required major interventions. The median time to any hospital intervention was 9.9 hours (IQR = 6.3 to 16.4 hours). We identified two independent predictors for hospital-level interventions: duration of symptoms > 24 hours (adjusted odds ratio [OR] = 2.1, 95% CI = 1.3 to 3.4) and location of the intussusception tip at (or proximal to) the hepatic flexure (adjusted OR = 1.9, 95% CI = 1.1 to 3.3); the latter factor was also a predictor of a major intervention. None of the children (95% CI = 0 to 1.0%) had an acute decompensation after an initially successful enema reduction.

CONCLUSIONS

Clinical decompensation is rare and recurrence is relatively low after an uncomplicated reduction of ileocolic intussusception. However, one in five children required hospital-level interventions after reduction. Children with the intussusception tip at (or proximal to) the hepatic flexure, and those with symptoms for longer than 24 hours, are more likely to require subsequent interventions. Although outpatient management appears safe after a period of observation, caregivers should be counseled about the risk of ongoing symptoms and recurrence.