Natural language processing with deep learning for medical adverse event detection from free-text medical narratives: A case study of detecting total hip replacement dislocation

Biosimilars in the Era of Artificial Intelligence-International Regulations and the Use in Oncological Treatments

This research is based on three fundamental aspects of successful biosimilar development in the challenging biopharmaceutical market. First, biosimilar regulations in eight selected countries: Japan, South Korea, the United States, Canada, Brazil, Argentina, Australia, and South Africa, represent the four continents. The regulatory aspects of the countries studied are analyzed, highlighting the challenges facing biosimilars, including their complex approval processes and the need for standardized regulatory guidelines. There is an inconsistency depending on whether the biosimilar is used in a developed or developing country. In the countries observed, biosimilars are considered excellent alternatives to patent-protected biological products for the treatment of chronic diseases. In the second aspect addressed, various analytical AI modeling methods (such as machine learning tools, reinforcement learning, supervised, unsupervised, and deep learning tools) were analyzed to observe patterns that lead to the prevalence of biosimilars used in cancer to model the behaviors of the most prominent active compounds with spectroscopy. Finally, an analysis of the use of active compounds of biosimilars used in cancer and approved by the FDA and EMA was proposed.

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data

Patient safety is a key quality issue for health systems. Healthcare acquired adverse events (AEs) compromise safety and quality; therefore, their reporting and monitoring is a patient safety priority. Although administrative datasets are potentially efficient tools for monitoring rates of AEs, concerns remain over the accuracy of their data. Chart review validation studies are required to explore the potential of administrative data to inform research and health policy. This review aims to present an overview of the methodological approaches and strategies used to validate rates of AEs in administrative data through chart review. This review was conducted in line with the Joanna Briggs Institute methodological framework for scoping reviews. Through database searches, 1054 sources were identified, imported into Covidence, and screened against the inclusion criteria. Articles that validated rates of AEs in administrative data through chart review were included. Data were extracted, exported to Microsoft Excel, arranged into a charting table, and presented in a tabular and descriptive format. Fifty-six studies were included. Most sources reported on surgical AEs; however, other medical specialties were also explored. Chart reviews were used in all studies; however, few agreed on terminology for the study design. Various methodological approaches and sampling strategies were used. Some studies used the Global Trigger Tool, a two-stage chart review method, whilst others used alternative single-, two-stage, or unclear approaches. The sources used samples of flagged charts (n = 24), flagged and random charts (n = 11), and random charts (n = 21). Most studies reported poor or moderate accuracy of AE rates. Some studies reported good accuracy of AE recording which highlights the potential of using administrative data for research purposes. This review highlights the potential for administrative data to provide information on AE rates and improve patient safety and healthcare quality. Nonetheless, further work is warranted to ensure that administrative data are accurate. The variation of methodological approaches taken, and sampling techniques used demonstrate a lack of consensus on best practice; therefore, further clarity and consensus are necessary to develop a more systematic approach to chart reviewing.

Application of deep learning for automated diagnosis and classification of hip dysplasia on plain radiographs

BACKGROUND

Hip dysplasia is a condition where the acetabulum is too shallow to support the femoral head and is commonly considered a risk factor for hip osteoarthritis. The objective of this study was to develop a deep learning model to diagnose hip dysplasia from plain radiographs and classify dysplastic hips based on their severity.

METHODS

We collected pelvic radiographs of 571 patients from two single-center cohorts and one multicenter cohort. The radiographs were split in half to create hip radiographs (n = 1022). One orthopaedic surgeon and one resident assessed the radiographs for hip dysplasia on either side. We used the center edge (CE) angle as the primary diagnostic criteria. Hips with a CE angle < 20°, 20° to 25°, and > 25° were labeled as dysplastic, borderline, and normal, respectively. The dysplastic hips were also classified with both Crowe and Hartofilakidis classification of dysplasia. The dataset was divided into train, validation, and test subsets using 80:10:10 split-ratio that were used to train two deep learning models to classify images into normal, borderline and (1) Crowe grade 1-4 or (2) Hartofilakidis grade 1-3. A pre-trained on Imagenet VGG16 convolutional neural network (CNN) was utilized by performing layer-wise fine-turning.

RESULTS

Both models struggled with distinguishing between normal and borderline hips. However, achieved high accuracy (Model 1: 92.2% and Model 2: 83.3%) in distinguishing between normal/borderline vs. dysplastic hips. The overall accuracy of Model 1 was 68% and for Model 2 73.5%. Most misclassifications for the Crowe and Hartofilakidis classifications were +/- 1 class from the correct class.

CONCLUSIONS

This pilot study shows promising results that a deep learning model distinguish between normal and dysplastic hips with high accuracy. Future research and external validation are warranted regarding the ability of deep learning models to perform complex tasks such as identifying and classifying disorders using plain radiographs.

LEVEL OF EVIDENCE

Diagnostic level IV.

Artificial intelligence in pediatric surgery

Artificial intelligence (AI) is rapidly changing the landscape of medicine and is already being utilized in conjunction with medical diagnostics and imaging analysis. We hereby explore AI applications in surgery and examine its relevance to pediatric surgery, covering its evolution, current state, and promising future. The various fields of AI are explored including machine learning and applications to predictive analytics and decision support in surgery, computer vision and image analysis in preoperative planning, image segmentation, surgical navigation, and finally, natural language processing assist in expediting clinical documentation, identification of clinical indications, quality improvement, outcome research, and other types of automated data extraction. The purpose of this review is to familiarize the pediatric surgical community with the rise of AI and highlight the ongoing advancements and challenges in its adoption, including data privacy, regulatory considerations, and the imperative for interdisciplinary collaboration. We hope this review serves as a comprehensive guide to AI's transformative influence on surgery, demonstrating its potential to enhance pediatric surgical patient outcomes, improve precision, and usher in a new era of surgical excellence.

The use of deep learning enables high diagnostic accuracy in detecting syndesmotic instability on weight-bearing CT scanning

PURPOSE

Delayed diagnosis of syndesmosis instability can lead to significant morbidity and accelerated arthritic change in the ankle joint. Weight-bearing computed tomography (WBCT) has shown promising potential for early and reliable detection of isolated syndesmotic instability using 3D volumetric measurements. While these measurements have been reported to be highly accurate, they are also experience-dependent, time-consuming, and need a particular 3D measurement software tool that leads the clinicians to still show more interest in the conventional diagnostic methods for syndesmotic instability. The purpose of this study was to increase accuracy, accelerate analysis time, and reduce interobserver bias by automating 3D volume assessment of syndesmosis anatomy using WBCT scans.

METHODS

A retrospective study was conducted using previously collected WBCT scans of patients with unilateral syndesmotic instability. One-hundred and forty-four bilateral ankle WBCT scans were evaluated (48 unstable, 96 control). We developed three deep learning models for analyzing WBCT scans to recognize syndesmosis instability. These three models included two state-of-the-art models (Model 1-3D Convolutional Neural Network [CNN], and Model 2-CNN with long short-term memory [LSTM]), and a new model (Model 3-differential CNN LSTM) that we introduced in this study.

RESULTS

Model 1 failed to analyze the WBCT scans (F1 score = 0). Model 2 only misclassified two cases (F1 score = 0.80). Model 3 outperformed Model 2 and achieved a nearly perfect performance, misclassifying only one case (F1 score = 0.91) in the control group as unstable while being faster than Model 2.

CONCLUSIONS

In this study, a deep learning model for 3D WBCT syndesmosis assessment was developed that achieved very high accuracy and accelerated analytics. This deep learning model shows promise for use by clinicians to improve diagnostic accuracy, reduce measurement bias, and save both time and expenditure for the healthcare system.

LEVEL OF EVIDENCE

II.

COVID-19 Vaccine Adverse Event Detection Based on Multi-Label Classification With Various Label Selection Strategies

Analyzing massive VAERS reports without medical context may lead to incorrect conclusions about vaccine adverse events (VAE). Facilitating VAE detection promotes continual safety improvement for new vaccines. This study proposes a multi-label classification method with various term-and topic-based label selection strategies to improve the accuracy and efficiency of VAE detection. Topic modeling methods are first used to generate rule-based label dependencies from Medical Dictionary for Regulatory Activities terms in VAE reports with two hyper-parameters. Multiple label selection strategies, namely one-vs-rest (OvsR), problem transformation (PT), algorithm adaption (AA), and deep learning (DL) methods, are used in multi-label classification to examine the model performance, respectively. Experimental results indicated that the topic-based PT methods improve the accuracy by up to 33.69% using a COVID-19 VAE reporting data set, which improves the robustness and interpretability of our models. In addition, the topic-based OvsR methods achieve an optimal accuracy of up to 98.88%. The accuracy of the AA methods with topic-based labels increased by up to 87.36%. By contrast, the state-of-art LSTM- and BERT-based DL methods have relatively poor performance with accuracy rates of 71.89% and 64.63%, respectively. Our findings reveal that the proposed method effectively improves the model accuracy and strengthens VAE interpretability by using different label selection strategies and domain knowledge in multi-label classification for VAE detection.

Developing an Inpatient Electronic Medical Record Phenotype for Hospital-Acquired Pressure Injuries: Case Study Using Natural Language Processing Models

BACKGROUND

Surveillance of hospital-acquired pressure injuries (HAPI) is often suboptimal when relying on administrative health data, as International Classification of Diseases (ICD) codes are known to have long delays and are undercoded. We leveraged natural language processing (NLP) applications on free-text notes, particularly the inpatient nursing notes, from electronic medical records (EMRs), to more accurately and timely identify HAPIs.

OBJECTIVE

This study aimed to show that EMR-based phenotyping algorithms are more fitted to detect HAPIs than ICD-10-CA algorithms alone, while the clinical logs are recorded with higher accuracy via NLP using nursing notes.

METHODS

Patients with HAPIs were identified from head-to-toe skin assessments in a local tertiary acute care hospital during a clinical trial that took place from 2015 to 2018 in Calgary, Alberta, Canada. Clinical notes documented during the trial were extracted from the EMR database after the linkage with the discharge abstract database. Different combinations of several types of clinical notes were processed by sequential forward selection during the model development. Text classification algorithms for HAPI detection were developed using random forest (RF), extreme gradient boosting (XGBoost), and deep learning models. The classification threshold was tuned to enable the model to achieve similar specificity to an ICD-based phenotyping study. Each model's performance was assessed, and comparisons were made between the metrics, including sensitivity, positive predictive value, negative predictive value, and F1-score.

RESULTS

Data from 280 eligible patients were used in this study, among whom 97 patients had HAPIs during the trial. RF was the optimal performing model with a sensitivity of 0.464 (95% CI 0.365-0.563), specificity of 0.984 (95% CI 0.965-1.000), and F1-score of 0.612 (95% CI of 0.473-0.751). The machine learning (ML) model reached higher sensitivity without sacrificing much specificity compared to the previously reported performance of ICD-based algorithms.

CONCLUSIONS

The EMR-based NLP phenotyping algorithms demonstrated improved performance in HAPI case detection over ICD-10-CA codes alone. Daily generated nursing notes in EMRs are a valuable data resource for ML models to accurately detect adverse events. The study contributes to enhancing automated health care quality and safety surveillance.

Development of an interactive web dashboard to facilitate the reexamination of pathology reports for instances of underbilling of CPT codes

Current Procedural Terminology Codes is a numerical coding system used to bill for medical procedures and services and crucially, represents a major reimbursement pathway. Given that pathology services represent a consequential source of hospital revenue, understanding instances where codes may have been misassigned or underbilled is critical. Several algorithms have been proposed that can identify improperly billed CPT codes in existing datasets of pathology reports. Estimation of the fiscal impacts of these reports requires a coder (i.e., billing staff) to review the original reports and manually code them again. As the re-assignment of codes using machine learning algorithms can be done quickly, the bottleneck in validating these reassignments is in this manual re-coding process, which can prove cumbersome. This work documents the development of a rapidly deployable dashboard for examination of reports that the original coder may have misbilled. Our dashboard features the following main components: (1) a bar plot to show the predicted probabilities for each CPT code, (2) an interpretation plot showing how each word in the report combines to form the overall prediction, and (3) a place for the user to input the CPT code they have chosen to assign. This dashboard utilizes the algorithms developed to accurately identify CPT codes to highlight the codes missed by the original coders. In order to demonstrate the function of this web application, we recruited pathologists to utilize it to highlight reports that had codes incorrectly assigned. We expect this application to accelerate the validation of re-assigned codes through facilitating rapid review of false-positive pathology reports. In the future, we will use this technology to review thousands of past cases in order to estimate the impact of underbilling has on departmental revenue.

Machine understanding surgical actions from intervention procedure textbooks

The automatic extraction of procedural surgical knowledge from surgery manuals, academic papers or other high-quality textual resources, is of the utmost importance to develop knowledge-based clinical decision support systems, to automatically execute some procedure's step or to summarize the procedural information, spread throughout the texts, in a structured form usable as a study resource by medical students. In this work, we propose a first benchmark on extracting detailed surgical actions from available intervention procedure textbooks and papers. We frame the problem as a Semantic Role Labeling task. Exploiting a manually annotated dataset, we apply different Transformer-based information extraction methods. Starting from RoBERTa and BioMedRoBERTa pre-trained language models, we first investigate a zero-shot scenario and compare the obtained results with a full fine-tuning setting. We then introduce a new ad-hoc surgical language model, named SurgicBERTa, pre-trained on a large collection of surgical materials, and we compare it with the previous ones. In the assessment, we explore different dataset splits (one in-domain and two out-of-domain) and we investigate also the effectiveness of the approach in a few-shot learning scenario. Performance is evaluated on three correlated sub-tasks: predicate disambiguation, semantic argument disambiguation and predicate-argument disambiguation. Results show that the fine-tuning of a pre-trained domain-specific language model achieves the highest performance on all splits and on all sub-tasks. All models are publicly released.

Concerns surrounding application of artificial intelligence in hip and knee arthroplasty

Literature surrounding artificial intelligence (AI)-related applications for hip and knee arthroplasty has proliferated. However, meaningful advances that fundamentally transform the practice and delivery of joint arthroplasty are yet to be realized, despite the broad range of applications as we continue to search for meaningful and appropriate use of AI. AI literature in hip and knee arthroplasty between 2018 and 2021 regarding image-based analyses, value-based care, remote patient monitoring, and augmented reality was reviewed. Concerns surrounding meaningful use and appropriate methodological approaches of AI in joint arthroplasty research are summarized. Of the 233 AI-related orthopaedics articles published, 178 (76%) constituted original research, while the rest consisted of editorials or reviews. A total of 52% of original AI-related research concerns hip and knee arthroplasty (n = 92), and a narrative review is described. Three studies were externally validated. Pitfalls surrounding present-day research include conflating vernacular (“AI/machine learning”), repackaging limited registry data, prematurely releasing internally validated prediction models, appraising model architecture instead of inputted data, withholding code, and evaluating studies using antiquated regression-based guidelines. While AI has been applied to a variety of hip and knee arthroplasty applications with limited clinical impact, the future remains promising if the question is meaningful, the methodology is rigorous and transparent, the data are rich, and the model is externally validated. Simple checkpoints for meaningful AI adoption include ensuring applications focus on: administrative support over clinical evaluation and management; necessity of the advanced model; and the novelty of the question being answered.

Cite this article: Bone Joint J 2022;104-B(12):1292–1303.

Multi-objective data enhancement for deep learning-based ultrasound analysis

Recently, Deep Learning based automatic generation of treatment recommendation has been attracting much attention. However, medical datasets are usually small, which may lead to over-fitting and inferior performances of deep learning models. In this paper, we propose multi-objective data enhancement method to indirectly scale up the medical data to avoid over-fitting and generate high quantity treatment recommendations. Specifically, we define a main and several auxiliary tasks on the same dataset and train a specific model for each of these tasks to learn different aspects of knowledge in limited data scale. Meanwhile, a Soft Parameter Sharing method is exploited to share learned knowledge among models. By sharing the knowledge learned by auxiliary tasks to the main task, the proposed method can take different semantic distributions into account during the training process of the main task. We collected an ultrasound dataset of thyroid nodules that contains Findings, Impressions and Treatment Recommendations labeled by professional doctors. We conducted various experiments on the dataset to validate the proposed method and justified its better performance than existing methods.

SafeMove: monitoring seniors with mild cognitive impairments using deep learning and location prediction

Due to society aging, age-related issues such as mild cognitive impairments (MCI) and dementia are attracting the attention of health professionals, scientists and governments. Seniors suffering from such impairments notice a slight deterioration in their cognitive abilities, which may lead to memory loss and behavioural disorders. In consequence, such seniors refrain from doing their everyday outdoor activities. Technology, e.g. smartphones, wearables and artificial intelligence, can provide seniors and their relatives with a variety of monitoring tools. In a nutshell, locations are analysed and, under specific situations, alarms are raised so that caregivers urgently informed. In this context, the discovery and prediction of trajectories and behaviours play a key role in deploying effective monitoring solutions. In this paper, we present a real-time smartphone-based monitoring system, called SafeMove, to discover and predict elderly people behaviours by analyzing outdoor trajectories. This is achieved by firstly analysing the elder’s mobility data previously collected using the proposed model called SpaceTime-Convolutional Neural Network (ST-CNN) in order to predict the most popular locations he/she might visit in the next time. Based on the predicted locations, the elder can be monitored in bounded region. Time and space-related variables, such as the distance traversed, the direction of the movements and the time spent, are analyzed in our abnormal behaviour detection (ABD) model that takes advantage of recurrent neural networks (RNNs). The effectiveness and the efficiency of our system for predicting the next location and detection the abnormal behaviors are evaluated using different datasets comprising real-world GPS trajectories.

Effects of Incontro, Alleanza, Responsabilita, Autonomia Intervention Model Combined with Orem Self-Care Model and the Use of Smart Wearable Devices on Perceived Stress and Self-Efficacy in Patients after Total Hip Arthroplasty

Objective

To explore the effects of Incontro, Alleanza, Responsabilita, Autonomia (IARA) combined with Orem self-care model and the use of smart wearable devices on perceived stress and self-efficacy in patients after total hip arthroplasty (THA).

Methods

A total of 60 patients after THA in our hospital were enrolled. Patients were randomly divided into control group (IARA intervention model combined with Orem self-care model) and study group (intelligent wearable device combined conference-IARA and Orem self-care model). Harris hip function score, Western Ontario and McMaster Universities Arthritis Index (WOMAC) score, functional independence measure (FIM) score, social support level, perceived stress, and self-efficacy were compared between the two groups.

Results

Harris hip function score, WOMAC score, FIM score, and the level of social support of the study group were higher compared with the control group after operation (P < 0.05). Additionally, the perceptual pressure in the study group was lower compared with the control group after intervention (P < 0.05). The self-efficacy of the two groups was compared, and the self-efficacy of the study group was higher than that of the control group at 4, 6, 8, and 12 weeks after the intervention, and the difference was statistically significant (P < 0.05).

Conclusion

Patients after THA utilize an intelligent wearable device combined with IARA model and Orem self-care model, which can effectively reduce awareness pressure, improve self-efficacy, and facilitate the improvement of the hip fracture.

Biophysical Model: A Promising Method in the Study of the Mechanism of Propofol: A Narrative Review

The physiological and neuroregulatory mechanism of propofol is largely based on very limited knowledge. It is one of the important puzzling issues in anesthesiology and is of great value in both scientific and clinical fields. It is acknowledged that neural networks which are comprised of a number of neural circuits might be involved in the anesthetic mechanism. However, the mechanism of this hypothesis needs to be further elucidated. With the progress of artificial intelligence, it is more likely to solve this problem through using artificial neural networks to perform temporal waveform data analysis and to construct biophysical computational models. This review focuses on current knowledge regarding the anesthetic mechanism of propofol, an intravenous general anesthetic, by constructing biophysical computational models.

Surgical reporting for laparoscopic cholecystectomy based on phase annotation by a convolutional neural network (CNN) and the phenomenon of phase flickering: a proof of concept

Purpose

Surgical documentation is an important yet time-consuming necessity in clinical routine. Beside its core function to transmit information about a surgery to other medical professionals, the surgical report has gained even more significance in terms of information extraction for scientific, administrative and judicial application. A possible basis for computer aided reporting is phase detection by convolutional neural networks (CNN). In this article we propose a workflow to generate operative notes based on the output of the TeCNO CNN.

Methods

Video recordings of 15 cholecystectomies were used for inference. The annotation of TeCNO was compared to that of an expert surgeon (HE) and the algorithm based annotation of a scientist (HA). The CNN output then was used to identify aberrance from standard course as basis for the final report. Moreover, we assessed the phenomenon of ‘phase flickering’ as clusters of incorrectly labeled frames and evaluated its usability.

Results

The accordance of the HE and CNN was 79.7% and that of HA and CNN 87.0%. ‘Phase flickering’ indicated an aberrant course with AUCs of 0.91 and 0.89 in ROC analysis regarding number and extend of concerned frames. Finally, we created operative notes based on a standard text, deviation alerts, and manual completion by the surgeon.

Conclusion

Computer-aided documentation is a noteworthy use case for phase recognition in standardized surgery. The analysis of phase flickering in a CNN’s annotation has the potential of retrieving more information about the course of a particular procedure to complement an automated report.

Prevalence of Sensitive Terms in Clinical Notes: observational study using natural language processing techniques (Preprint)

Background

Objective

Methods

Results

Conclusions

Machine learning and artificial intelligence in research and healthcare✰,✰✰

Artificial intelligence (AI) is a broad term referring to the application of computational algorithms that can analyze large data sets to classify, predict, or gain useful conclusions. Under the umbrella of AI is machine learning (ML). ML is the process of building or learning statistical models using previously observed real world data to predict outcomes, or categorize observations based on 'training' provided by humans. These predictions are then applied to future data, all the while folding in the new data into its perpetually improving and calibrated statistical model. The future of AI and ML in healthcare research is exciting and expansive. AI and ML are becoming cornerstones in the medical and healthcare-research domains and are integral in our continued processing and capitalization of robust patient EMR data. Considerations for the use and application of ML in healthcare settings include assessing the quality of data inputs and decision-making that serve as the foundations of the ML model, ensuring the end-product is interpretable, transparent, and ethical concerns are considered throughout the development process. The current and future applications of ML include improving the quality and quantity of data collected from EMRs to improve registry data, utilizing these robust datasets to improve and standardized research protocols and outcomes, clinical decision-making applications, natural language processing and improving the fundamentals of value-based care, to name only a few.

Natural Language Processing and Its Use in Orthopaedic Research

PURPOSE OF REVIEW

This review aims to demonstrate how natural language processing is used in orthopaedic research.

RECENT FINDINGS

Natural language processing is a form of artificial intelligence that involves encoding human-generated text or speech into a form which can be interpreted by computers to perform a variety of tasks. Natural language processing gathers, processes, and organizes large amounts of free-text data more efficiently than humans. In orthopaedics, it has been utilized for retrospective chart review, automated reporting of electronic health record data, analyzing operative notes and radiology reports, and patient reviews of physicians and practices. Although still in its infancy, natural language processing promises to be a valuable tool in the future of orthopaedic research. It will not eliminate the need for the essential human component of questioning involved in research, but natural language processing can improve the quality, efficiency, and thoroughness of research, thus improving patient care.

Identifying Patients with Hypoglycemia Using Natural Language Processing: A Systematic Literature Review (Preprint)

Background

Accurately identifying patients with hypoglycemia is key to preventing adverse events and mortality. Natural language processing (NLP), a form of artificial intelligence, uses computational algorithms to extract information from text data. NLP is a scalable, efficient, and quick method to extract hypoglycemia-related information when using electronic health record data sources from a large population.

Objective

The objective of this systematic review was to synthesize the literature on the application of NLP to extract hypoglycemia from electronic health record clinical notes.

Methods

Literature searches were conducted electronically in PubMed, Web of Science Core Collection, CINAHL (EBSCO), PsycINFO (Ovid), IEEE Xplore, Google Scholar, and ACL Anthology. Keywords included hypoglycemia, low blood glucose, NLP, and machine learning. Inclusion criteria included studies that applied NLP to identify hypoglycemia, reported the outcomes related to hypoglycemia, and were published in English as full papers.

Results

This review (n=8 studies) revealed heterogeneity of the reported results related to hypoglycemia. Of the 8 included studies, 4 (50%) reported that the prevalence rate of any level of hypoglycemia was 3.4% to 46.2%. The use of NLP to analyze clinical notes improved the capture of undocumented or missed hypoglycemic events using International Classification of Diseases, Ninth Revision (ICD-9), and International Classification of Diseases, Tenth Revision (ICD-10), and laboratory testing. The combination of NLP and ICD-9 or ICD-10 codes significantly increased the identification of hypoglycemic events compared with individual methods; for example, the prevalence rates of hypoglycemia were 12.4% for International Classification of Diseases codes, 25.1% for an NLP algorithm, and 32.2% for combined algorithms. All the reviewed studies applied rule-based NLP algorithms to identify hypoglycemia.

Conclusions

The findings provided evidence that the application of NLP to analyze clinical notes improved the capture of hypoglycemic events, particularly when combined with the ICD-9 or ICD-10 codes and laboratory testing.

Automatic Classification of Thyroid Findings Using Static and Contextualized Ensemble Natural Language Processing Systems: Development Study

Background

In the case of Korean institutions and enterprises that collect nonstandardized and nonunified formats of electronic medical examination results from multiple medical institutions, a group of experienced nurses who can understand the results and related contexts initially classified the reports manually. The classification guidelines were established by years of workers’ clinical experiences and there were attempts to automate the classification work. However, there have been problems in which rule-based algorithms or human labor–intensive efforts can be time-consuming or limited owing to high potential errors. We investigated natural language processing (NLP) architectures and proposed ensemble models to create automated classifiers.

Objective

This study aimed to develop practical deep learning models with electronic medical records from 284 health care institutions and open-source corpus data sets for automatically classifying 3 thyroid conditions: healthy, caution required, and critical. The primary goal is to increase the overall accuracy of the classification, yet there are practical and industrial needs to correctly predict healthy (negative) thyroid condition data, which are mostly medical examination results, and minimize false-negative rates under the prediction of healthy thyroid conditions.

Methods

The data sets included thyroid and comprehensive medical examination reports. The textual data are not only documented in fully complete sentences but also written in lists of words or phrases. Therefore, we propose static and contextualized ensemble NLP network (SCENT) systems to successfully reflect static and contextual information and handle incomplete sentences. We prepared each convolution neural network (CNN)-, long short-term memory (LSTM)-, and efficiently learning an encoder that classifies token replacements accurately (ELECTRA)-based ensemble model by training or fine-tuning them multiple times. Through comprehensive experiments, we propose 2 versions of ensemble models, SCENT-v1 and SCENT-v2, with the single-architecture–based CNN, LSTM, and ELECTRA ensemble models for the best classification performance and practical use, respectively. SCENT-v1 is an ensemble of CNN and ELECTRA ensemble models, and SCENT-v2 is a hierarchical ensemble of CNN, LSTM, and ELECTRA ensemble models. SCENT-v2 first classifies the 3 labels using an ELECTRA ensemble model and then reclassifies them using an ensemble model of CNN and LSTM if the ELECTRA ensemble model predicted them as “healthy” labels.

Results

SCENT-v1 outperformed all the suggested models, with the highest F1 score (92.56%). SCENT-v2 had the second-highest recall value (94.44%) and the fewest misclassifications for caution-required thyroid condition while maintaining 0 classification error for the critical thyroid condition under the prediction of the healthy thyroid condition.

Conclusions

The proposed SCENT demonstrates good classification performance despite the unique characteristics of the Korean language and problems of data lack and imbalance, especially for the extremely low amount of critical condition data. The result of SCENT-v1 indicates that different perspectives of static and contextual input token representations can enhance classification performance. SCENT-v2 has a strong impact on the prediction of healthy thyroid conditions.