An Introduction to Natural Language Processing: How You Can Get More From Those Electronic Notes You Are Generating

Surveillance of Health Care-Associated Violence Using Natural Language Processing

BACKGROUND AND OBJECTIVES

Patient and family violent outbursts toward staff, caregivers, or through self-harm, have increased during the ongoing behavioral health crisis. These health care-associated violence (HAV) episodes are likely under-reported. We sought to assess the feasibility of using nursing notes to identify under-reported HAV episodes.

METHODS

We extracted nursing notes across inpatient units at 2 hospitals for 2019: a pediatric tertiary care center and a community-based hospital. We used a workflow for narrative data processing using a natural language processing (NLP) assisted manual review process performed by domain experts (a nurse and a physician). We trained the NLP models on the tertiary care center data and validated it on the community hospital data. Finally, we applied these surveillance methods to real-time data for 2022 to assess reporting completeness of new cases.

RESULTS

We used 70 981 notes from the tertiary care center for model building and internal validation and 19 332 notes from the community hospital for external validation. The final community hospital model sensitivity was 96.8% (95% CI 90.6% to 100%) and a specificity of 47.1% (39.6% to 54.6%) compared with manual review. We identified 31 HAV episodes in July to December 2022, of which 26 were reportable in accordance with the hospital internal criteria. Only 7 of 26 cases were reported by employees using the self-reporting system, all of which were identified by our surveillance process.

CONCLUSIONS

NLP-assisted review is a feasible method for surveillance of under-reported HAV episodes, with implementation and usability that can be achieved even at a low information technology-resourced hospital setting.

Machine learning/artificial intelligence in sports medicine: state of the art and future directions

Machine learning (ML) is changing the way health care is practiced and recent applications of these novel statistical techniques have started to impact orthopaedic sports medicine. Machine learning enables the analysis of large volumes of data to establish complex relationships between "input" and "output" variables. These relationships may be more complex than could be established through traditional statistical analysis and can lead to the ability to predict the "output" with high levels of accuracy. Supervised learning is the most common ML approach for healthcare data and recent studies have developed algorithms to predict patient-specific outcome after surgical procedures such as hip arthroscopy and anterior cruciate ligament reconstruction. Deep learning is a higher-level ML approach that facilitates the processing and interpretation of complex datasets through artificial neural networks that are inspired by the way the human brain processes information. In orthopaedic sports medicine, deep learning has primarily been used for automatic image (computer vision) and text (natural language processing) interpretation. While applications in orthopaedic sports medicine have been increasing exponentially, one significant barrier to widespread adoption of ML remains clinician unfamiliarity with the associated methods and concepts. The goal of this review is to introduce these concepts, review current machine learning models in orthopaedic sport medicine, and discuss future opportunities for innovation within the specialty.

Enhancing Pressure Injury Surveillance Using Natural Language Processing

OBJECTIVE

This study assessed the feasibility of nursing handoff notes to identify underreported hospital-acquired pressure injury (HAPI) events.

METHODS

We have established a natural language processing-assisted manual review process and workflow for data extraction from a corpus of nursing notes across all medical inpatient and intensive care units in a tertiary care pediatric center. This system is trained by 2 domain experts. Our workflow started with keywords around HAPI and treatments, then regular expressions, distributive semantics, and finally a document classifier. We generated 3 models: a tri-gram classifier, binary logistic regression model using the regular expressions as predictors, and a random forest model using both models together. Our final output presented to the event screener was generated using a random forest model validated using derivation and validation sets.

RESULTS

Our initial corpus involved 70,981 notes during a 1-year period from 5484 unique admissions for 4220 patients. Our interrater human reviewer agreement on identifying HAPI was high ( κ = 0.67; 95% confidence interval [CI], 0.58-0.75). Our random forest model had 95% sensitivity (95% CI, 90.6%-99.3%), 71.2% specificity (95% CI, 65.1%-77.2%), and 78.7% accuracy (95% CI, 74.1%-83.2%). A total of 264 notes from 148 unique admissions (2.7% of all admissions) were identified describing likely HAPI. Sixty-one described new injuries, and 64 describe known yet possibly evolving injuries. Relative to the total patient population during our study period, HAPI incidence was 11.9 per 1000 discharges, and incidence rate was 1.2 per 1000 bed-days.

CONCLUSIONS

Natural language processing-based surveillance is proven to be feasible and high yield using nursing handoff notes.

Artificial Intelligence Models in Health Information Exchange: A Systematic Review of Clinical Implications

Electronic health record (EHR) systems collate patient data, and the integration and standardization of documents through Health Information Exchange (HIE) play a pivotal role in refining patient management. Although the clinical implications of AI in EHR systems have been extensively analyzed, its application in HIE as a crucial source of patient data is less explored. Addressing this gap, our systematic review delves into utilizing AI models in HIE, gauging their predictive prowess and potential limitations. Employing databases such as Scopus, CINAHL, Google Scholar, PubMed/Medline, and Web of Science and adhering to the PRISMA guidelines, we unearthed 1021 publications. Of these, 11 were shortlisted for the final analysis. A noticeable preference for machine learning models in prognosticating clinical results, notably in oncology and cardiac failures, was evident. The metrics displayed AUC values ranging between 61% and 99.91%. Sensitivity metrics spanned from 12% to 96.50%, specificity from 76.30% to 98.80%, positive predictive values varied from 83.70% to 94.10%, and negative predictive values between 94.10% and 99.10%. Despite variations in specific metrics, AI models drawing on HIE data unfailingly showcased commendable predictive proficiency in clinical verdicts, emphasizing the transformative potential of melding AI with HIE. However, variations in sensitivity highlight underlying challenges. As healthcare's path becomes more enmeshed with AI, a well-rounded, enlightened approach is pivotal to guarantee the delivery of trustworthy and effective AI-augmented healthcare solutions.

Can Natural Language Processing and Artificial Intelligence Automate The Generation of Billing Codes From Operative Note Dictations?

STUDY DESIGN

Retrospective Cohort Study.

OBJECTIVES

Using natural language processing (NLP) in combination with machine learning on standard operative notes may allow for efficient billing, maximization of collections, and minimization of coder error. This study was conducted as a pilot study to determine if a machine learning algorithm can accurately identify billing Current Procedural Terminology (CPT) codes on patient operative notes.

METHODS

This was a retrospective analysis of operative notes from patients who underwent elective spine surgery by a single senior surgeon from 9/2015 to 1/2020. Algorithm performance was measured by performing receiver operating characteristic (ROC) analysis, calculating the area under the ROC curve (AUC) and the area under the precision-recall curve (AUPRC). A deep learning NLP algorithm and a Random Forest algorithm were both trained and tested on operative notes to predict CPT codes. CPT codes generated by the billing department were compared to those generated by our model.

RESULTS

The random forest machine learning model had an AUC of .94 and an AUPRC of .85. The deep learning model had a final AUC of .72 and an AUPRC of .44. The random forest model had a weighted average, class-by-class accuracy of 87%. The LSTM deep learning model had a weighted average, class-by-class accuracy 0f 59%.

CONCLUSIONS

Combining natural language processing with machine learning is a valid approach for automatic generation of CPT billing codes. The random forest machine learning model outperformed the LSTM deep learning model in this case. These models can be used by orthopedic or neurosurgery departments to allow for efficient billing.

Inter-rater agreement for the annotation of neurologic signs and symptoms in electronic health records

The extraction of patient signs and symptoms recorded as free text in electronic health records is critical for precision medicine. Once extracted, signs and symptoms can be made computable by mapping to signs and symptoms in an ontology. Extracting signs and symptoms from free text is tedious and time-consuming. Prior studies have suggested that inter-rater agreement for clinical concept extraction is low. We have examined inter-rater agreement for annotating neurologic concepts in clinical notes from electronic health records. After training on the annotation process, the annotation tool, and the supporting neuro-ontology, three raters annotated 15 clinical notes in three rounds. Inter-rater agreement between the three annotators was high for text span and category label. A machine annotator based on a convolutional neural network had a high level of agreement with the human annotators but one that was lower than human inter-rater agreement. We conclude that high levels of agreement between human annotators are possible with appropriate training and annotation tools. Furthermore, more training examples combined with improvements in neural networks and natural language processing should make machine annotators capable of high throughput automated clinical concept extraction with high levels of agreement with human annotators.

Natural language processing in radiology: Clinical applications and future directions

Natural language processing (NLP) is a wide range of techniques that allows computers to interact with human text. Applications of NLP in everyday life include language translation aids, chat bots, and text prediction. It has been increasingly utilized in the medical field with increased reliance on electronic health records. As findings in radiology are primarily communicated via text, the field is particularly suited to benefit from NLP based applications. Furthermore, rapidly increasing imaging volume will continue to increase burden on clinicians, emphasizing the need for improvements in workflow. In this article, we highlight the numerous non-clinical, provider focused, and patient focused applications of NLP in radiology. We also comment on challenges associated with development and incorporation of NLP based applications in radiology as well as potential future directions.

Natural language processing: using artificial intelligence to understand human language in orthopedics

Natural language processing (NLP) describes the broad field of artificial intelligence by which computers are trained to understand and generate human language. Within healthcare research, NLP is commonly used for variable extraction and classification/cohort identification tasks. While these tools are becoming increasingly popular and available as both open-source and commercial products, there is a paucity of the literature within the orthopedic space describing the key tasks within these powerful pipelines. Curation and navigation of the electronic medical record are becoming increasingly onerous, and it is important for physicians and other healthcare professionals to understand potential methods of harnessing this large data resource. The purpose of this study is to provide an overview of the tasks required to develop an NLP pipeline for orthopedic research and present recent examples of successful implementations.

Electronic surveillance of patient safety events using natural language processing

Objective

We describe our approach to surveillance of reportable safety events captured in hospital data including free-text clinical notes. We hypothesize that a) some patient safety events are documented only in the clinical notes and not in any other accessible source; and b) large-scale abstraction of event data from clinical notes is feasible.

Materials and Methods

We use regular expressions to generate a training data set for a machine learning model and apply this model to the full set of clinical notes and conduct further review to identify safety events of interest. We demonstrate this approach on peripheral intravenous (PIV) infiltrations and extravasations (PIVIEs).

Results

During Phase 1, we collected 21,362 clinical notes, of which 2342 were reviewed. We identified 125 PIV events, of which 44 cases (35%) were not captured by other patient safety systems. During Phase 2, we collected 60,735 clinical notes and identified 440 infiltrate events. Our classifier demonstrated accuracy above 90%.

Conclusion

Our method to identify safety events from the free text of clinical documentation offers a feasible and scalable approach to enhance existing patient safety systems. Expert reviewers, using a machine learning model, can conduct routine surveillance of patient safety events.

Text Score Analysis under the IPE Environment Based on Improved Transformer

In order to improve the accuracy of ideological and political education (IPE) text scoring, an improved short-text similarity calculation model based on transformer is proposed. This model takes the DSSM model as the basic framework and uses the Bert model to realize text representation and solve polysemy problem. The transformer encoding component is used to extract the characteristics of the text and obtain the internal information of the text. With the help of the encoding component, the two texts can interact with information on multiple levels. Finally, the semantic similarity between two texts is calculated by concatenation vector inference.

Assessment of Natural Language Processing Methods for Ascertaining the Expanded Disability Status Scale Score From the Electronic Health Records of Patients With Multiple Sclerosis: Algorithm Development and Validation Study

Background

The Expanded Disability Status Scale (EDSS) score is a widely used measure to monitor disability progression in people with multiple sclerosis (MS). However, extracting and deriving the EDSS score from unstructured electronic health records can be time-consuming.

Objective

We aimed to compare rule-based and deep learning natural language processing algorithms for detecting and predicting the total EDSS score and EDSS functional system subscores from the electronic health records of patients with MS.

Methods

We studied 17,452 electronic health records of 4906 MS patients followed at one of Canada’s largest MS clinics between June 2015 and July 2019. We randomly divided the records into training (80%) and test (20%) data sets, and compared the performance characteristics of 3 natural language processing models. First, we applied a rule-based approach, extracting the EDSS score from sentences containing the keyword “EDSS.” Next, we trained a convolutional neural network (CNN) model to predict the 19 half-step increments of the EDSS score. Finally, we used a combined rule-based–CNN model. For each approach, we determined the accuracy, precision, recall, and F-score compared with the reference standard, which was manually labeled EDSS scores in the clinic database.

Results

Overall, the combined keyword-CNN model demonstrated the best performance, with accuracy, precision, recall, and an F-score of 0.90, 0.83, 0.83, and 0.83 respectively. Respective figures for the rule-based and CNN models individually were 0.57, 0.91, 0.65, and 0.70, and 0.86, 0.70, 0.70, and 0.70. Because of missing data, the model performance for EDSS subscores was lower than that for the total EDSS score. Performance improved when considering notes with known values of the EDSS subscores.

Conclusions

A combined keyword-CNN natural language processing model can extract and accurately predict EDSS scores from patient records. This approach can be automated for efficient information extraction in clinical and research settings.

The Food and Drug Administration Biologics Effectiveness and Safety Initiative Facilitates Detection of Vaccine Administrations From Unstructured Data in Medical Records Through Natural Language Processing

Introduction: The Food and Drug Administration Center for Biologics Evaluation and Research conducts post-market surveillance of biologic products to ensure their safety and effectiveness. Studies have found that common vaccine exposures may be missing from structured data elements of electronic health records (EHRs), instead being captured in clinical notes. This impacts monitoring of adverse events following immunizations (AEFIs). For example, COVID-19 vaccines have been regularly administered outside of traditional medical settings. We developed a natural language processing (NLP) algorithm to mine unstructured clinical notes for vaccinations not captured in structured EHR data.

Methods: A random sample of 1,000 influenza vaccine administrations, representing 995 unique patients, was extracted from a large U.S. EHR database. NLP techniques were used to detect administrations from the clinical notes in the training dataset [80% (N = 797) of patients]. The algorithm was applied to the validation dataset [20% (N = 198) of patients] to assess performance. Full medical charts for 28 randomly selected administration events in the validation dataset were reviewed by clinicians. The NLP algorithm was then applied across the entire dataset (N = 995) to quantify the number of additional events identified.

Results: A total of 3,199 administrations were identified in the structured data and clinical notes combined. Of these, 2,740 (85.7%) were identified in the structured data, while the NLP algorithm identified 1,183 (37.0%) administrations in clinical notes; 459 were not also captured in the structured data. This represents a 16.8% increase in the identification of vaccine administrations compared to using structured data alone. The validation of 28 vaccine administrations confirmed 27 (96.4%) as “definite” vaccine administrations; 18 (64.3%) had evidence of a vaccination event in the structured data, while 10 (35.7%) were found solely in the unstructured notes.

Discussion: We demonstrated the utility of an NLP algorithm to identify vaccine administrations not captured in structured EHR data. NLP techniques have the potential to improve detection of vaccine administrations not otherwise reported without increasing the analysis burden on physicians or practitioners. Future applications could include refining estimates of vaccine coverage and detecting other exposures, population characteristics, and outcomes not reliably captured in structured EHR data.

Decreasing delivery room CPAP-associated pneumothorax at ≥35-week gestational age

OBJECTIVE

We previously reported an increase in pneumothorax after implementing delivery room (DR) continuous positive airway pressure (CPAP) for labored breathing or persistent cyanosis in ≥35-week gestational age (GA) neonates unexposed to DR-positive pressure ventilation (DR-PPV). We hypothesized that pneumothorax would decrease after de-implementing DR-CPAP in those unexposed to DR-PPV or DR-O₂ supplementation (DR-PPV/O₂).

STUDY DESIGN

In a retrospective cohort excluding DR-PPV the primary outcome was DR-CPAP-related pneumothorax (1st chest radiogram, 1st day of life). In a subgroup treated by the resuscitation team and admitted to the NICU, the primary outcome was DR-CPAP-associated pneumothorax (1st radiogram, no prior PPV) without DR-PPV/O₂.

RESULTS

In the full cohort, occurrence of DR-CPAP-related pneumothorax decreased after the intervention (11.0% vs 6.0%, P < 0.001). In the subgroup, occurrence of DR-CPAP-associated pneumothorax decreased after the intervention (1.4% vs. 0.06%, P < 0.001).

CONCLUSION

The occurrence of CPAP-associated pneumothorax decreased after avoiding DR-CPAP in ≥35-week GA neonates without DR-PPV/O₂.

Natural Language Processing Applications in the Clinical Neurosciences: A Machine Learning Augmented Systematic Review

Natural language processing (NLP), a domain of artificial intelligence (AI) that models human language, has been used in medicine to automate diagnostics, detect adverse events, support decision making and predict clinical outcomes. However, applications to the clinical neurosciences appear to be limited. NLP has matured with the implementation of deep transformer models (e.g., XLNet, BERT, T5, and RoBERTa) and transfer learning. The objectives of this study were to (1) systematically review NLP applications in the clinical neurosciences, and (2) explore NLP analysis to facilitate literature synthesis, providing clear examples to demonstrate the potential capabilities of these technologies for a clinical audience. Our NLP analysis consisted of keyword identification, text summarization and document classification. A total of 48 articles met inclusion criteria. NLP has been applied in the clinical neurosciences to facilitate literature synthesis, data extraction, patient identification, automated clinical reporting and outcome prediction. The number of publications applying NLP has increased rapidly over the past five years. Document classifiers trained to differentiate included and excluded articles demonstrated moderate performance (XLNet AUC = 0.66, BERT AUC = 0.59, RoBERTa AUC = 0.62). The T5 transformer model generated acceptable abstract summaries. The application of NLP has the potential to enhance research and practice in the clinical neurosciences.

Utility of Blood Cultures and Empiric Antibiotics in Febrile Pediatric Hemophilia Patients With Central Venous Access Devices

BACKGROUND

Children with hemophilia frequently require long-term central venous access devices (CVADs) for regular infusion of factor products. Hemophilia patients are not immunocompromised, but the presence and use of CVADs are associated with infections including bacteremia. Currently, the utility of blood cultures in evaluation of the febrile hemophilia patient with an indwelling CVAD is unknown, nor is optimal empiric antibiotic use.

METHODS

We performed a retrospective cross-sectional study of febrile immunocompetent hemophilia patients with CVADs presenting to a large academic urban pediatric emergency department from 1995 to 2017. We used a natural language processing electronic search, followed by manual chart review to construct the cohort. We analyzed rate of pathogen recovery from cultures of blood in subgroups of hemophilia patients, the pathogen profile, and the reported pathogen susceptibilities to ceftriaxone.

RESULTS

Natural language processing electronic search identified 181 visits for fever among hemophilia patients with indwelling CVADs of which 147 cases from 44 unique patients met study criteria. Cultures of blood were positive in 56 (38%) of 147 patients (95% confidence interval, 30%-47%). Seventeen different organisms were isolated (10 pathogens and 7 possible pathogens) with Staphylococcus aureus and coagulase-negative Staphylococcus species as the most common. Thirty-four percent of isolates were reported as susceptible to ceftriaxone. Positive blood cultures were more common in cases involving patients with inhibitors (n = 71) versus those without (n = 76), odds ratio, 7.4 (95% confidence interval, 3.5-15.9). This was observed irrespective of hemophilia type.

CONCLUSIONS

Febrile immunocompetent hemophilia patients with indwelling CVADs have high rates of bacteremia. Empiric antimicrobial therapy should be targeted to anticipated pathogens and take into consideration local susceptibility patterns for Staphylococcus aureus.

Linking Free Text Documentation of Functioning and Disability to the ICF With Natural Language Processing

Background: Invaluable information on patient functioning and the complex interactions that define it is recorded in free text portions of the Electronic Health Record (EHR). Leveraging this information to improve clinical decision-making and conduct research requires natural language processing (NLP) technologies to identify and organize the information recorded in clinical documentation.

Methods: We used natural language processing methods to analyze information about patient functioning recorded in two collections of clinical documents pertaining to claims for federal disability benefits from the U.S. Social Security Administration (SSA). We grounded our analysis in the International Classification of Functioning, Disability, and Health (ICF), and used the Activities and Participation domain of the ICF to classify information about functioning in three key areas: mobility, self-care, and domestic life. After annotating functional status information in our datasets through expert clinical review, we trained machine learning-based NLP models to automatically assign ICF categories to mentions of functional activity.

Results: We found that rich and diverse information on patient functioning was documented in the free text records. Annotation of 289 documents for Mobility information yielded 2,455 mentions of Mobility activities and 3,176 specific actions corresponding to 13 ICF-based categories. Annotation of 329 documents for Self-Care and Domestic Life information yielded 3,990 activity mentions and 4,665 specific actions corresponding to 16 ICF-based categories. NLP systems for automated ICF coding achieved over 80% macro-averaged F-measure on both datasets, indicating strong performance across all ICF categories used.

Conclusions: Natural language processing can help to navigate the tradeoff between flexible and expressive clinical documentation of functioning and standardizable data for comparability and learning. The ICF has practical limitations for classifying functional status information in clinical documentation but presents a valuable framework for organizing the information recorded in health records about patient functioning. This study advances the development of robust, ICF-based NLP technologies to analyze information on patient functioning and has significant implications for NLP-powered analysis of functional status information in disability benefits management, clinical care, and research.

Natural language processing and machine learning to assist radiation oncology incident learning

PURPOSE

To develop a Natural Language Processing (NLP) and Machine Learning (ML) pipeline that can be integrated into an Incident Learning System (ILS) to assist radiation oncology incident learning by semi-automating incident classification. Our goal was to develop ML models that can generate label recommendations, arranged according to their likelihoods, for three data elements in Canadian NSIR-RT taxonomy.

METHODS

Over 6000 incident reports were gathered from the Canadian national ILS as well as our local ILS database. Incident descriptions from these reports were processed using various NLP techniques. The processed data with the expert-generated labels were used to train and evaluate over 500 multi-output ML algorithms. The top three models were identified and tuned for each of three different taxonomy data elements, namely: (1) process step where the incident occurred, (2) problem type of the incident and (3) the contributing factors of the incident. The best-performing model after tuning was identified for each data element and tested on unseen data.

RESULTS

The MultiOutputRegressor extended Linear SVR models performed best on the three data elements. On testing, our models ranked the most appropriate label 1.48 ± 0.03, 1.73 ± 0.05 and 2.66 ± 0.08 for process-step, problem-type and contributing factors respectively.

CONCLUSIONS

We developed NLP-ML models that can perform incident classification. These models will be integrated into our ILS to generate a drop-down menu. This semi-automated feature has the potential to improve the usability, accuracy and efficiency of our radiation oncology ILS.

Is lymphangitic streaking associated with different pathogens?

OBJECTIVES

Little is known regarding the differences in microbiology associated with cellulitis or abscess with or without lymphangitic streaking. The objective of our study is to assess whether there are differences in the pathogens identified from wound cultures of patients with paronychia with and without associated lymphangitis.

METHODS

Retrospective cross-sectional study at a tertiary pediatric emergency department over 25 years. We opted to assess patients with paronychia of the finger, assuming that these cases will have a greater variety of causative pathogens compared to other cases of cellulitis and soft tissue abscess that are associated with nail biting. Case identification was conducted using a computerized text-screening search that was refined by manual chart review. We included patients from 1 month to 20 years of age who underwent an incision and drainage (I&D) of a paronychia and had a culture obtained. The presence or absence of lymphangitis was determined from the clinical narrative in the medical record. We excluded patients treated with antibiotics prior to I&D as well as immune-compromised patients. We used descriptive statistics for prevalence and χ2 tests for categorical variables.

RESULTS

Two hundred sixty-six patients met inclusion criteria. The median age was 9.7 years [IQR 4.7, 15.4] and 45.1% were female. Twenty-two patients (8.3%) had lymphangitic streaking associated with their paronychia. Patients with lymphangitis streaking were similar to those without lymphangitis in terms of age and sex (p = 0.52 and p = 0.82, respectively). Overall, the predominant bacteria was MSSA (40%) followed by MRSA (26%). No significant differences were found between the pathogens in the 22 patients with associated lymphangitis compared to the 244 patients without.

CONCLUSION

Staphylococcus aureus represent the majority of pathogens in paronychia, although streptococcal species and gram-negative bacteria were also common. Among patients with paronychia of the finger, there seems to be no association between pathogen type and presence of lymphangitic streaking.

Racial and Ethnic Disparities in Pain Management of Children With Limb Fractures or Suspected Appendicitis: A Retrospective Cross-Sectional Study

Objective: To evaluate whether racial/ethnical differences in analgesia administration existed in two different cohorts of children with painful conditions: children with either limb fracture or suspected appendicitis. Methods: Retrospective cross-sectional analysis of children visiting a pediatric emergency department (Boston Children Hospital) for limb fracture or suspected appendicitis from 2011 to 2015. We computed the proportion of children that received any analgesic treatment and any opioid analgesia. We performed multivariable logistic regressions to investigate race/ethnicity differences in analgesic and opioid administration, after adjusting for pain score, demographics and visit covariates. Results: Among the 8,347 children with a limb fracture and the 4,780 with suspected appendicitis, 65.0 and 60.9% received any analgesic treatment, and 35.9 and 33.4% an opioid analgesia, respectively. Compared to White non-Hispanic Children, Black non-Hispanic children and Hispanic children were less likely to receive opioid analgesia in both the limb fracture cohort [Black: aOR = 0.61 (95% CI, 0.50-0.75); Hispanic aOR = 0.66 (95% CI, 0.55-0.80)] and in the suspected appendicitis cohort [Black: aOR = 0.75 (95% CI, 0.58-0.96); Hispanic aOR = 0.78 (95% CI, 0.63-0.96)]. In the limb fracture cohort, Black non-Hispanic children and Hispanic children were more likely to receive any analgesic treatment (non-opioid or opioid) than White non-Hispanic children [Black: aOR = 1.63 (95% CI, 1.33-2.01); Hispanic aOR = 1.43 (95% CI, 1.19-1.72)]. Conclusion: Racial and ethnic disparities exist in the pain management of two different painful conditions, which suggests true inequities in health care delivery. To provide equitable analgesic care, emergency departments should monitor variation in analgesic management and develop appropriate universal interventions.

Presentation, Diagnostic Evaluation, Management, and Rates of Serious Bacterial Infection in Infants With Acute Dacryocystitis Presenting to the Emergency Department

BACKGROUND

Dacryocystitis is considered benign, yet infants represent a population at risk of complications. The presentation, management, and rates of serious bacterial infection in infants with dacryocystitis have not been described.

METHODS

We conducted a retrospective study of infants (12 months or younger) presenting to a single urban tertiary care pediatric emergency department between January 1995 and March 2014 with concern for dacryocystitis. Exclusion criteria included immune compromise or craniofacial anomalies. Cases were identified using text search software, followed by manual chart review.

RESULTS

We identified 333 subjects, and median age was 38 days (interquartile range, 12; 106). Fifty-three percent were female. Most were afebrile (81%, T < 38°C) at triage while 6% had fever of ≥39°C. Two of 135 blood cultures sent were positive (both Streptococcus pneumoniae). Lumbar punctures were performed on 40 patients (12%), and no cerebrospinal fluid (CSF) cultures were positive. Eye cultures were positive in 47% (N = 58) of infants cultured (N = 123); the most common pathogens were Haemophilus species (N = 17), Staphylococcus aureus (N = 13), Gram-negative rods (N = 7), and Moraxella species (N = 4). Imaging was obtained in 11 subjects (3.3%) with 3 demonstrating cellulitis and 1 a hemangioma. Ophthalmology was consulted for 21%, and an intervention/probe performed in 6%. Topical antibiotics were used in 147 subjects (44%), oral antibiotics in 100 (33%), and parenteral antibiotics in 87 (26%).

CONCLUSION

Infants with dacryocystitis have a variable presentation and management ranges from observation to aggressive management. The rates of serious bacterial infection were low in this sample and not associated with any presenting risk factors.

Bacteriology of pediatric breast abscesses beyond the neonatal period

BACKGROUND

Limited data exist regarding the presentation and bacteriology of nonneonatal pediatric breast abscess.

OBJECTIVE

To determine the bacteriology and characteristic presentation of pediatric breast abscesses in a tertiary care center.

METHODS

Cross-sectional study of patients age 1 month to 21 years admitted to a pediatric Emergency Department (ED) between 1996 and 2018 with a breast abscess. Patients with pre-existing conditions were excluded. Records were reviewed to determine demographics, history, physical exam findings, wound culture results, imaging and ED disposition. We used descriptive statistics to describe prevalence of different bacteria.

RESULTS

We identified 210 patients who met study criteria. Median age was 13.6 years [IQR 6.6, 17.4], and 91% (191/210) were females. Ninety-two patients (43.8%) were 'pre-treated' with antibiotics prior to ED visit, and 33/210 (16%) were febrile. Ultrasound was obtained in 85 patients (40.5%), 69 patients had a single abscess and 16 had multiple abscesses. Most patients were treated with antibiotics and 100 had a surgical intervention, of these 89 had I&D and 11 a needle aspiration. Admission rate was 45%. Culture results were available for 75 (75%). Thirty-three (44%) had a negative culture, or grew non-aureus staphylococci or other skin flora. Culture were positive for MSSA 21 (28%), MRSA 13 (17%), Proteus mirabilis 2 (2.6%), Serratia 1 (1.3%). Other organisms include Gram-negative bacilli, group A Streptococcus and enterococcus.

CONCLUSIONS

Non-neonatal pediatric breast abscess bacteriology is no different than data published on other skin abscesses. MRSA coverage should be considered based on local prevalence in skin infections.

Improving the prediction of streptococcal pharyngitis; time to move past exudate alone

BACKGROUND

Palatal petechiae are predictive of Group A streptococcal (GAS) pharyngitis. We sought to (a) quantify the value of considering petechiae in addition to exudate, and (b) assess provider incorporation of petechiae's predictive nature for GAS into clinical decision making.

METHODS

We conducted a cross-sectional study of patients 3-21 years with sore throat and GAS testing performed in a pediatric emergency department (ED) in 2016. Patients were excluded if immunosuppressed, nonverbal, medically complex, had chronic tonsillitis, or received antibiotics in the preceding week. As a proxy of provider incorporation of petechiae into clinical decision making we assessed how often petechiae were documented, compared with exudate. We performed univariate analysis using χ² analysis for categorical data and Mann-Whitney U test for continuous data.

RESULTS

1574 patients met inclusion criteria. Median age 8 years [IQR 5, 13]; 54% female. 372 patients (24%) were GAS positive. Both palatal petechiae and tonsillar exudates were predictive of GAS [OR 8.5 (95% CI 5.2-13.9), and 1.9 (95% CI 1.4-2.6) respectively]. Examining petechiae or exudate vs. exudate alone increases OR from 1.9 to 2.9 (95% CI 2.2-3.8). Sensitivity improves (23% to 34%) with minimal change to specificity (87% to 85%). Among those with a normal or erythematous throat exam, petechiae were mentioned as a pertinent negative in 28%; absence of tonsillar exudate was mentioned in 78% (p = .02).

CONCLUSIONS

Palatal petechiae are highly associated with GAS, yet rarely addressed in documentation. Incorporating palatal petechiae into common scoring systems could improve prediction and disseminate this knowledge into practice.

Automatic Incident Triage in Radiation Oncology Incident Learning System

The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports.

Enhancing predictions of patient conveyance using emergency call handler free text notes for unconscious and fainting incidents reported to the London Ambulance Service

OBJECTIVE

Pre-hospital emergency medical services use clinical decision support systems (CDSS) to triage calls. Call handlers often supplement this by making free text notes covering key incident information. We investigate whether machine learning approaches using features from such free text notes can improve prediction of unconscious patients who require conveyance.

MATERIALS AND METHODS

We analysed a subset of all London Ambulance Service calls that were triaged through the Medical Priority Dispatch System (MPDS) as involving an unconscious or fainting patient in 2018. We use and compare two machine learning algorithms: random forest (RF) and gradient boosting machine (GBM). For each incident, we predict whether the patient will be conveyed to a hospital emergency department or equivalent using as features 1) the MPDS code, 2) the free text notes and 3) the two together. We evaluate model performance using the area under the curve (AUC) metric. Given the imbalance of outcomes (patient conveyed 71 %, not conveyed 29 %), we also consider sensitivity and specificity.

RESULTS

Using only the MPDS code resulted in an AUC of 0.57. Using the text notes gave an improved AUC score of 0.63 and combining the two gave an AUC score of 0.64 (scores were similar for RF and GBM). GBM models scored better on sensitivity (0.93 vs 0.62 for RF in the combined model), but specificity was lower (0.17 vs. 0.56 for RF in the combined model).

CONCLUSIONS

Using information contained in the free text notes made by call handlers in combination with MPDS improves prediction of unconscious and fainting patients requiring conveyance to a hospital emergency department (or equivalent) when compared with machine learning models using MPDS codes only. This suggests there is some useful information in unstructured data captured by emergency call handlers that complements MPDS codes. Quantifying this gain can help inform emergency medical service policy when evaluating the decision to expand or augment existing CDSS.

Artificial intelligence approaches using natural language processing to advance EHR-based clinical research

The wide adoption of electronic health record systems in health care generates big real-world data that open new venues to conduct clinical research. As a large amount of valuable clinical information is locked in clinical narratives, natural language processing techniques as an artificial intelligence approach have been leveraged to extract information from clinical narratives in electronic health records. This capability of natural language processing potentially enables automated chart review for identifying patients with distinctive clinical characteristics in clinical care and reduces methodological heterogeneity in defining phenotype, obscuring biological heterogeneity in research concerning allergy, asthma, and immunology. This brief review discusses the current literature on the secondary use of electronic health record data for clinical research concerning allergy, asthma, and immunology and highlights the potential, challenges, and implications of natural language processing techniques.

Impact of viral symptoms on the performance of the modified centor score to predict pediatric group A streptococcal pharyngitis

BACKGROUND

Clinicians use the Modified Centor Score (MCS) to estimate the risk of group A streptococcal (GAS) pharyngitis in children with sore throat. The Infectious Diseases Society of America (IDSA) recommends neither testing nor treating patients with specific viral symptoms. The goal of this study is to measure the impact of those symptoms on the yield of GAS testing predicted by the MCS.

METHODS

Retrospective cohort study of all patients aged 3-21 years presenting with sore throat and tested for GAS in a pediatric emergency department (ED) in 2016. After identifying all patients tested for GAS, we used natural language processing (NLP) to identify the subgroup complaining of sore throat. We abstracted all MCS variables as well as symptoms suggestive of a viral etiology per the IDSA guideline (conjunctivitis, coryza, cough, diarrhea, hoarseness, ulcerative oral lesions, viral exanthema). We calculated the proportion of patients who tested positive for GAS by MCS with and without viral symptoms.

RESULTS

Of the 1574 patients included, 372 patients (24%) tested GAS positive. Patients with at least one viral symptom had a reduced GAS risk compared to those without any of the viral symptoms 91/547 (17% GAS positive) vs. 281/1027 (27%), odds ratio 0.53 (95% CI 0.41-0.69).

CONCLUSIONS

The presence of viral symptoms specified by the IDSA alters the predicted yield of testing by traditional MCS. Clinicians may consider adjusting interpretation of a patient's MCS based on the presence of viral symptoms, but viral symptoms may not always fully obviate the need for GAS testing.

A Review of Automatic Phenotyping Approaches using Electronic Health Records

Electronic Health Records (EHR) are a rich repository of valuable clinical information that exist in primary and secondary care databases. In order to utilize EHRs for medical observational research a range of algorithms for automatically identifying individuals with a specific phenotype have been developed. This review summarizes and offers a critical evaluation of the literature relating to studies conducted into the development of EHR phenotyping systems. This review describes phenotyping systems and techniques based on structured and unstructured EHR data. Articles published on PubMed and Google scholar between 2013 and 2017 have been reviewed, using search terms derived from Medical Subject Headings (MeSH). The popularity of using Natural Language Processing (NLP) techniques in extracting features from narrative text has increased. This increased attention is due to the availability of open source NLP algorithms, combined with accuracy improvement. In this review, Concept extraction is the most popular NLP technique since it has been used by more than 50% of the reviewed papers to extract features from EHR. High-throughput phenotyping systems using unsupervised machine learning techniques have gained more popularity due to their ability to efficiently and automatically extract a phenotype with minimal human effort.

Potential Impact of Initial Clinical Data on Adjustment of Pediatric Readmission Rates

OBJECTIVE

Comparison of readmission rates requires adjustment for case-mix (ie, differences in patient populations), but previously only claims data were available for this purpose. We examined whether incorporation of relatively readily available clinical data improves prediction of pediatric readmissions and thus might enhance case-mix adjustment.

METHODS

We examined 30-day readmissions using claims and electronic health record data for patients ≤18 years and 29 days of age who were admitted to 3 children's hospitals from February 2011 to February 2014. Using the Pediatric All-Condition Readmission Measure and starting with a model including age, gender, chronic conditions, and primary diagnosis, we examined whether the addition of initial vital sign and laboratory data improved model performance. We employed machine learning to evaluate the same variables, using the L2-regularized logistic regression with cost-sensitive learning and convolutional neural network.

RESULTS

Controlling for the core model variables, low red blood cell count and mean corpuscular hemoglobin concentration and high red cell distribution width were associated with greater readmission risk, as were certain interactions between laboratory and chronic condition variables. However, the C-statistic (0.722 vs 0.713) and McFadden's pseudo R² (0.085 vs 0.076) for this and the core model were similar, suggesting minimal improvement in performance. In machine learning analyses, the F-measure (harmonic mean of sensitivity and positive predictive value) was similar for the best-performing model (containing all variables) and core model (0.250 vs 0.243).

CONCLUSIONS

Readily available clinical variables do not meaningfully improve the prediction of pediatric readmissions and would be unlikely to enhance case-mix adjustment unless their distributions varied widely across hospitals.

Automated detection of sudden unexpected death in epilepsy risk factors in electronic medical records using natural language processing

OBJECTIVE

Sudden unexpected death in epilepsy (SUDEP) is an important cause of mortality in epilepsy. However, there is a gap in how often providers counsel patients about SUDEP. One potential solution is to electronically prompt clinicians to provide counseling via automated detection of risk factors in electronic medical records (EMRs). We evaluated (1) the feasibility and generalizability of using regular expressions to identify risk factors in EMRs and (2) barriers to generalizability.

METHODS

Data included physician notes for 3000 patients from one medical center (home) and 1000 from five additional centers (away). Through chart review, we identified three SUDEP risk factors: (1) generalized tonic-clonic seizures, (2) refractory epilepsy, and (3) epilepsy surgery candidacy. Regular expressions of risk factors were manually created with home training data, and performance was evaluated with home test and away test data. Performance was evaluated by sensitivity, positive predictive value, and F-measure. Generalizability was defined as an absolute decrease in performance by <0.10 for away versus home test data. To evaluate underlying barriers to generalizability, we identified causes of errors seen more often in away data than home data. To demonstrate how small revisions can improve generalizability, we removed three "boilerplate" standard text phrases from away notes and repeated performance.

RESULTS

We observed high performance in home test data (F-measure range = 0.86-0.90), and low to high performance in away test data (F-measure range = 0.53-0.81). After removing three boilerplate phrases, away performance improved (F-measure range = 0.79-0.89) and generalizability was achieved for nearly all measures. The only significant barrier to generalizability was use of boilerplate phrases, causing 104 of 171 errors (61%) in away data.

SIGNIFICANCE

Regular expressions are a feasible and probably a generalizable method to identify variables related to SUDEP risk. Our methods may be implemented to create large patient cohorts for research and to generate electronic prompts for SUDEP counseling.

Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches

Learning from patient safety incident reports is a vital part of improving healthcare. However, the volume of reports and their largely free-text nature poses a major analytic challenge. The objective of this study was to test the capability of autonomous classifying of free text within patient safety incident reports to determine incident type and the severity of harm outcome. Primary care patient safety incident reports (n=31333) previously expert-categorised by clinicians (training data) were processed using J48, SVM and Naïve Bayes.The SVM classifier was the highest scoring classifier for incident type (AUROC, 0.891) and severity of harm (AUROC, 0.708). Incident reports containing deaths were most easily classified, correctly identifying 72.82% of reports. In conclusion, supervised ML can be used to classify patient safety incident report categories. The severity classifier, whilst not accurate enough to replace manual processing, could provide a valuable screening tool for this critical aspect of patient safety.

Predictors of Primary Intracranial Hypertension in Children Using a Newly Suggested Opening Pressure Cutoff of 280 mm H2O

OBJECTIVES

We assessed the clinical characteristics of primary intracranial hypertension (PIH) in children using a newly recommended threshold for cerebrospinal fluid opening pressure (280 mm H₂O).

METHOD

Cross-sectional study of patients age ≤21 years who had a lumbar puncture done for evaluation of PIH. Patients were excluded if lumbar puncture was done for a suspected infection, seizure, mental status changes, multiple sclerosis, or Guillain-Barre syndrome. Cases were identified using a text-search module followed by manual review. We performed χ2 analysis for categorical data and Mann-Whitney U test for continuous data, followed by a binary logistic regression.

RESULTS

We identified 374 patients of whom 67% were female, median age was 13 years interquartile range (11 to 16 years), and admission rate was 24%. Using an opening pressure cutoff of 250 mm H₂O, 127 patients (34%) were identified as having PIH, whereas using the new cutoff 105 patients (28%) met PIH criteria. Predictors for PIH included optic disc edema or sixth nerve palsy using both old, odds ratio (OR) 7.6 (4.3, 13.5), and new cutoffs, OR 9.7 (95% confidence interval 5.1, 18.5). Headache duration ≤61 days is predictive of PIH using the new cutoff OR 4.1 (95% confidence interval 1.3, 12.8). A model is presented which stratifies patients into groups with low (7%), medium (18%), and high (greater than 42%) risk of PIH.

CONCLUSIONS

A higher cerebrospinal fluid opening pressure threshold in the criteria of PIH is associated with PIH patients with a different symptom profile. Children with optic disc edema, bulging fontanel or sixth nerve palsy, are at increased risk for PIH.

Priorities to Overcome Barriers Impacting Data Science Application in Emergency Care Research

For a variety of reasons including cheap computing, widespread adoption of electronic medical records, digitalization of imaging and biosignals, and rapid development of novel technologies, the amount of health care data being collected, recorded, and stored is increasing at an exponential rate. Yet despite these advances, methods for the valid, efficient, and ethical utilization of these data remain underdeveloped. Emergency care research, in particular, poses several unique challenges in this rapidly evolving field. A group of content experts was recently convened to identify research priorities related to barriers to the application of data science to emergency care research. These recommendations included: 1) developing methods for cross-platform identification and linkage of patients; 2) creating central, deidentified, open-access databases; 3) improving methodologies for visualization and analysis of intensively sampled data; 4) developing methods to identify and standardize electronic medical record data quality; 5) improving and utilizing natural language processing; 6) developing and utilizing syndrome or complaint-based based taxonomies of disease; 7) developing practical and ethical framework to leverage electronic systems for controlled trials; 8) exploring technologies to help enable clinical trials in the emergency setting; and 9) training emergency care clinicians in data science and data scientists in emergency care medicine. The background, rationale, and conclusions of these recommendations are included in the present article.

Pediatric Emergency Department Visits for Homelessness After Shelter Eligibility Policy Change

BACKGROUND AND OBJECTIVES

In 2012, Massachusetts changed its emergency shelter eligibility policy for homeless families. One new criterion to document homelessness was staying in a location "not meant for human habitation," and the emergency department (ED) fulfilled this requirement. Our aim for this study is to analyze the frequency and costs of pediatric ED visits for homelessness before and after this policy.

METHODS

This is a retrospective study of ED visits for homelessness at a children's hospital from March 2010 to February 2016. A natural language processing tool was used to identify cases, which were manually reviewed for inclusion. We compared demographic and homelessness circumstance characteristics and conducted an interrupted time series analysis to compare ED visits by homeless children before and after the policy. We compared the change in ED visits for homelessness to the number of homeless children in Massachusetts. We analyzed payment data for each visit.

RESULTS

There were 312 ED visits for homelessness; 95% (n = 297) of visits were after the policy. These visits increased 4.5 times after the policy (95% confidence interval: 1.33 to 15.23). Children seen after the policy were more likely to have no medical complaint (rate ratio: 3.27; 95% confidence interval: 1.18 to 9.01). Although the number of homeless children in Massachusetts increased 1.4 times over the study period, ED visits for homelessness increased 13-fold. Payments (average: $557 per visit) were >4 times what a night in a shelter would cost; 89% of payments were made through state-based insurance plans.

CONCLUSIONS

A policy change to Massachusetts' shelter eligibility was associated with increased pediatric ED visits for homelessness along with substantial health care costs.

Validation of a Natural Language Processing Algorithm for Detecting Infectious Disease Symptoms in Primary Care Electronic Medical Records in Singapore

Background

Free-text clinical records provide a source of information that complements traditional disease surveillance. To electronically harness these records, they need to be transformed into codified fields by natural language processing algorithms.

Objective

The aim of this study was to develop, train, and validate Clinical History Extractor for Syndromic Surveillance (CHESS), an natural language processing algorithm to extract clinical information from free-text primary care records.

Methods

CHESS is a keyword-based natural language processing algorithm to extract 48 signs and symptoms suggesting respiratory infections, gastrointestinal infections, constitutional, as well as other signs and symptoms potentially associated with infectious diseases. The algorithm also captured the assertion status (affirmed, negated, or suspected) and symptom duration.

Electronic medical records from the National Healthcare Group Polyclinics, a major public sector primary care provider in Singapore, were randomly extracted and manually reviewed by 2 human reviewers, with a third reviewer as the adjudicator. The algorithm was evaluated based on 1680 notes against the human-coded result as the reference standard, with half of the data used for training and the other half for validation.

Results

The symptoms most commonly present within the 1680 clinical records at the episode level were those typically present in respiratory infections such as cough (744/7703, 9.66%), sore throat (591/7703, 7.67%), rhinorrhea (552/7703, 7.17%), and fever (928/7703, 12.04%). At the episode level, CHESS had an overall performance of 96.7% precision and 97.6% recall on the training dataset and 96.0% precision and 93.1% recall on the validation dataset. Symptoms suggesting respiratory and gastrointestinal infections were all detected with more than 90% precision and recall. CHESS correctly assigned the assertion status in 97.3%, 97.9%, and 89.8% of affirmed, negated, and suspected signs and symptoms, respectively (97.6% overall accuracy). Symptom episode duration was correctly identified in 81.2% of records with known duration status.

Conclusions

We have developed an natural language processing algorithm dubbed CHESS that achieves good performance in extracting signs and symptoms from primary care free-text clinical records. In addition to the presence of symptoms, our algorithm can also accurately distinguish affirmed, negated, and suspected assertion statuses and extract symptom durations.

Predictors of a drainable suppurative adenitis among children presenting with cervical adenopathy

OBJECTIVES

We sought to identify predictors for a drainable suppurative adenitis [DSA] among patients presenting with acute cervical lymphadenitis.

METHODS

A retrospective cross sectional study of all patients admitted to an urban pediatric tertiary care emergency department over a 15 year period. Otherwise healthy patients who underwent imaging for an evaluation of cervical lymphadenitis were included. Cases were identified using a text-search module followed by manual review. We excluded immunocompromised patients and those with lymphadenopathy felt to be not directly infected (i.e. reactive) or that was not acute (symptom duration >28 days). Data collected included: age, gender, duration of symptoms, highest recorded temperature, physical exam findings, laboratory and imaging results, and surgical findings. A DSA was defined as >1.5 cm in diameter on imaging. We performed binary logistic regression to determine independent clinical predictors of a DSA.

RESULTS

Three hundred sixty-one patients met inclusion criteria. Three hundred six patients (85%) had a CT scan, 55 (15%) had an ultrasound and 33 (9%) had both. DSA was identified in 71 (20%) patients. Clinical features independently associated with a DSA included absence of clinical pharyngitis, WBC >15,000/mm³, age ≤3 years, anterior cervical chain location, largest palpable diameter on exam >3 cm and prior antibiotic treatment of >24 h. The presence of fever, skin erythema, or fluctuance on examination, was not found to be predictive of DSA.

CONCLUSIONS

We identified independent predictors of DSA among children presenting with cervical adenitis. Risk can be stratified into risk groups based on these clinical features.

Can a collaborative healthcare network improve the care of people with epilepsy?

New opportunities are now available to improve care in ways not possible previously. Information contained in electronic medical records can now be shared without identifying patients. With network collaboration, large numbers of medical records can be searched to identify patients most like the one whose complex medical situation challenges the physician. The clinical effectiveness of different treatment strategies can be assessed rapidly to help the clinician decide on the best treatment for this patient. Other capabilities from different components of the network can prompt the recognition of what is the best available option and encourage the sharing of information about programs and electronic tools. Difficulties related to privacy, harmonization, integration, and costs are expected, but these are currently being addressed successfully by groups of organizations led by those who recognize the benefits.

Automatic Lung-RADS™ classification with a natural language processing system

Background

Our aim was to train a natural language processing (NLP) algorithm to capture imaging characteristics of lung nodules reported in a structured CT report and suggest the applicable Lung-RADS™ (LR) category.

Methods

Our study included structured, clinical reports of consecutive CT lung screening (CTLS) exams performed from 08/2014 to 08/2015 at an ACR accredited Lung Cancer Screening Center. All patients screened were at high-risk for lung cancer according to the NCCN Guidelines^®. All exams were interpreted by one of three radiologists credentialed to read CTLS exams using LR using a standard reporting template. Training and test sets consisted of consecutive exams. Lung screening exams were divided into two groups: three training sets (500, 120, and 383 reports each) and one final evaluation set (498 reports). NLP algorithm results were compared with the gold standard of LR category assigned by the radiologist.

Results

The sensitivity/specificity of the NLP algorithm to correctly assign LR categories for suspicious nodules (LR 4) and positive nodules (LR 3/4) were 74.1%/98.6% and 75.0%/98.8% respectively. The majority of mismatches occurred in cases where pulmonary findings were present not currently addressed by LR. Misclassifications also resulted from the failure to identify exams as follow-up and the failure to completely characterize part-solid nodules. In a sub-group analysis among structured reports with standardized language, the sensitivity and specificity to detect LR 4 nodules were 87.0% and 99.5%, respectively.

Conclusions

An NLP system can accurately suggest the appropriate LR category from CTLS exam findings when standardized reporting is used.

Diagnostic Lumbar Puncture Among Children With Facial Palsy in a Lyme Disease Endemic Area

We identified 620 children with peripheral facial palsy of which 211 (34%) had Lyme disease. The 140 children who had a lumbar puncture performed were more likely to be hospitalized (73% LP performed vs 2% no LP) and to receive parenteral antibiotics (62% LP performed vs 6% no LP).

Validation of the Total Visual Acuity Extraction Algorithm (TOVA) for Automated Extraction of Visual Acuity Data From Free Text, Unstructured Clinical Records

PURPOSE

With increasing volumes of electronic health record data, algorithm-driven extraction may aid manual extraction. Visual acuity often is extracted manually in vision research. The total visual acuity extraction algorithm (TOVA) is presented and validated for automated extraction of visual acuity from free text, unstructured clinical notes.

METHODS

Consecutive inpatient ophthalmology notes over an 8-year period from the University of Washington healthcare system in Seattle, WA were used for validation of TOVA. The total visual acuity extraction algorithm applied natural language processing to recognize Snellen visual acuity in free text notes and assign laterality. The best corrected measurement was determined for each eye and converted to logMAR. The algorithm was validated against manual extraction of a subset of notes.

RESULTS

A total of 6266 clinical records were obtained giving 12,452 data points. In a subset of 644 validated notes, comparison of manually extracted data versus TOVA output showed 95% concordance. Interrater reliability testing gave κ statistics of 0.94 (95% confidence interval [CI], 0.89-0.99), 0.96 (95% CI, 0.94-0.98), 0.95 (95% CI, 0.92-0.98), and 0.94 (95% CI, 0.90-0.98) for acuity numerators, denominators, adjustments, and signs, respectively. Pearson correlation coefficient was 0.983. Linear regression showed an R² of 0.966 (P < 0.0001).

CONCLUSIONS

The total visual acuity extraction algorithm is a novel tool for extraction of visual acuity from free text, unstructured clinical notes and provides an open source method of data extraction.

TRANSLATIONAL RELEVANCE

Automated visual acuity extraction through natural language processing can be a valuable tool for data extraction from free text ophthalmology notes.

How Much Cerebrospinal Fluid Should We Remove Prior to Measuring a Closing Pressure?

OBJECTIVE

The objective of this study was to identify a relationship between cerebrospinal fluid (CSF) volume removal and change in CSF pressure in children with suspected idiopathic intracranial hypertension (IIH).

METHODS

We performed a cross-sectional study of children 22 years and younger who underwent a lumbar puncture (LP) and had a documented opening pressure, closing pressure, and volume removed. Relationship between volume removal and pressure change was determined using a fractional polynomial regression procedure.

RESULTS

In the 297 patients who met the inclusion criteria, CSF pressure decreased by 1 cm H₂O for every 0.91 mL of CSF removed if the maximum change in pressure was less than 15 cm H₂O ( R² = 0.38).

CONCLUSION

A linear relationship exists between the volume of CSF removed and the amount of pressure relieved when the desired pressure change is less than 15 cm H₂O.

Natural Language Processing-Enabled and Conventional Data Capture Methods for Input to Electronic Health Records: A Comparative Usability Study

Background

The process of documentation in electronic health records (EHRs) is known to be time consuming, inefficient, and cumbersome. The use of dictation coupled with manual transcription has become an increasingly common practice. In recent years, natural language processing (NLP)–enabled data capture has become a viable alternative for data entry. It enables the clinician to maintain control of the process and potentially reduce the documentation burden. The question remains how this NLP-enabled workflow will impact EHR usability and whether it can meet the structured data and other EHR requirements while enhancing the user’s experience.

Objective

The objective of this study is evaluate the comparative effectiveness of an NLP-enabled data capture method using dictation and data extraction from transcribed documents (NLP Entry) in terms of documentation time, documentation quality, and usability versus standard EHR keyboard-and-mouse data entry.

Methods

This formative study investigated the results of using 4 combinations of NLP Entry and Standard Entry methods (“protocols”) of EHR data capture. We compared a novel dictation-based protocol using MediSapien NLP (NLP-NLP) for structured data capture against a standard structured data capture protocol (Standard-Standard) as well as 2 novel hybrid protocols (NLP-Standard and Standard-NLP). The 31 participants included neurologists, cardiologists, and nephrologists. Participants generated 4 consultation or admission notes using 4 documentation protocols. We recorded the time on task, documentation quality (using the Physician Documentation Quality Instrument, PDQI-9), and usability of the documentation processes.

Results

A total of 118 notes were documented across the 3 subject areas. The NLP-NLP protocol required a median of 5.2 minutes per cardiology note, 7.3 minutes per nephrology note, and 8.5 minutes per neurology note compared with 16.9, 20.7, and 21.2 minutes, respectively, using the Standard-Standard protocol and 13.8, 21.3, and 18.7 minutes using the Standard-NLP protocol (1 of 2 hybrid methods). Using 8 out of 9 characteristics measured by the PDQI-9 instrument, the NLP-NLP protocol received a median quality score sum of 24.5; the Standard-Standard protocol received a median sum of 29; and the Standard-NLP protocol received a median sum of 29.5. The mean total score of the usability measure was 36.7 when the participants used the NLP-NLP protocol compared with 30.3 when they used the Standard-Standard protocol.

Conclusions

In this study, the feasibility of an approach to EHR data capture involving the application of NLP to transcribed dictation was demonstrated. This novel dictation-based approach has the potential to reduce the time required for documentation and improve usability while maintaining documentation quality. Future research will evaluate the NLP-based EHR data capture approach in a clinical setting. It is reasonable to assert that EHRs will increasingly use NLP-enabled data entry tools such as MediSapien NLP because they hold promise for enhancing the documentation process and end-user experience.

A comparison between physicians and computer algorithms for form CMS-2728 data reporting

INTRODUCTION

CMS-2728 form (Medical Evidence Report) assesses 23 comorbidities chosen to reflect poor outcomes and increased mortality risk. Previous studies questioned the validity of physician reporting on forms CMS-2728. We hypothesize that reporting of comorbidities by computer algorithms identifies more comorbidities than physician completion, and, therefore, is more reflective of underlying disease burden.

METHODS

We collected data from CMS-2728 forms for all 296 patients who had incident ESRD diagnosis and received chronic dialysis from 2005 through 2014 at Indiana University outpatient dialysis centers. We analyzed patients' data from electronic medical records systems that collated information from multiple health care sources. Previously utilized algorithms or natural language processing was used to extract data on 10 comorbidities for a period of up to 10 years prior to ESRD incidence. These algorithms incorporate billing codes, prescriptions, and other relevant elements. We compared the presence or unchecked status of these comorbidities on the forms to the presence or absence according to the algorithms.

FINDINGS

Computer algorithms had higher reporting of comorbidities compared to forms completion by physicians. This remained true when decreasing data span to one year and using only a single health center source. The algorithms determination was well accepted by a physician panel. Importantly, algorithms use significantly increased the expected deaths and lowered the standardized mortality ratios.

DISCUSSION

Using computer algorithms showed superior identification of comorbidities for form CMS-2728 and altered standardized mortality ratios. Adapting similar algorithms in available EMR systems may offer more thorough evaluation of comorbidities and improve quality reporting.