Using deep learning and natural language processing models to detect child physical abuse

Precision emergency medicine

BACKGROUND

Precision health is a burgeoning scientific discipline that aims to incorporate individual variability in biological, behavioral, and social factors to develop personalized health solutions. To date, emergency medicine has not deeply engaged in the precision health movement. However, rapid advances in health technology, data science, and medical informatics offer new opportunities for emergency medicine to realize the promises of precision health.

METHODS

In this article, we conceptualize precision emergency medicine as an emerging paradigm and identify key drivers of its implementation into current and future clinical practice. We acknowledge important obstacles to the specialty-wide adoption of precision emergency medicine and offer solutions that conceive a successful path forward.

RESULTS

Precision emergency medicine is defined as the use of information and technology to deliver acute care effectively, efficiently, and authentically to individual patients and their communities. Key drivers and opportunities include leveraging human data, capitalizing on technology and digital tools, providing deliberate access to care, advancing population health, and reimagining provider education and roles. Overcoming challenges in equity, privacy, and cost is essential for success. We close with a call to action to proactively incorporate precision health into the clinical practice of emergency medicine, the training of future emergency physicians, and the research agenda of the specialty.

CONCLUSIONS

Precision emergency medicine leverages new technology and data-driven artificial intelligence to advance diagnostic testing, individualize patient care plans and therapeutics, and strategically refine the convergence of the health system and the community.

Bidirectional Encoder Representations from Transformers in Radiology: A Systematic Review of Natural Language Processing Applications

INTRODUCTION

Bidirectional Encoder Representations from Transformers (BERT), introduced in 2018, has revolutionized natural language processing. Its bidirectional understanding of word context has enabled innovative applications, notably in radiology. This study aimed to assess BERT's influence and applications within the radiologic domain.

METHODS

Adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic review, searching PubMed for literature on BERT-based models and natural language processing in radiology from January 1, 2018, to February 12, 2023. The search encompassed keywords related to generative models, transformer architecture, and various imaging techniques.

RESULTS

Of 597 results, 30 met our inclusion criteria. The remaining were unrelated to radiology or did not use BERT-based models. The included studies were retrospective, with 14 published in 2022. The primary focus was on classification and information extraction from radiology reports, with x-rays as the prevalent imaging modality. Specific investigations included automatic CT protocol assignment and deep learning applications in chest x-ray interpretation.

CONCLUSION

This review underscores the primary application of BERT in radiology for report classification. It also reveals emerging BERT applications for protocol assignment and report generation. As BERT technology advances, we foresee further innovative applications. Its implementation in radiology holds potential for enhancing diagnostic precision, expediting report generation, and optimizing patient care.

ChatGPT Yields a Passing Score on a Pediatric Board Preparatory Exam but Raises Red Flags

Objectives

We aimed to evaluate the performance of a publicly-available online artificial intelligence program (OpenAI's ChatGPT-3.5 and -4.0, August 3 versions) on a pediatric board preparatory examination, 2021 and 2022 PREP® Self-Assessment, American Academy of Pediatrics (AAP).

Methods

We entered 245 questions and answer choices from the Pediatrics 2021 PREP® Self-Assessment and 247 questions and answer choices from the Pediatrics 2022 PREP® Self-Assessment into OpenAI's ChatGPT-3.5 and ChatGPT-4.0, August 3 versions, in September 2023. The ChatGPT-3.5 and 4.0 scores were compared with the advertised passing scores (70%+) for the PREP® exams and the average scores (74.09%) and (75.71%) for all 10 715 and 6825 first-time human test takers.

Results

For the AAP 2021 and 2022 PREP® Self-Assessments, ChatGPT-3.5 answered 143 of 243 (58.85%) and 137 of 247 (55.46%) questions correctly on a single attempt. ChatGPT-4.0 answered 193 of 243 (79.84%) and 208 of 247 (84.21%) questions correctly.

Conclusion

Using a publicly-available online chatbot to answer pediatric board preparatory examination questions yielded a passing score but demonstrated significant limitations in the chatbot's ability to assess some complex medical situations in children, posing a potential risk to this vulnerable population.

A chemical accident cause text mining method based on improved accident triangle

BACKGROUND

With the rapid development of China's chemical industry, although researchers have developed many methods in the field of chemical safety, the situation of chemical safety in China is still not optimistic. How to prevent accidents has always been the focus of scholars' attention.

METHODS

Based on the characteristics of chemical enterprises and the Heinrich accident triangle, this paper developed the organizational-level accident triangle, which divides accidents into group-level, unit-level, and workshop-level accidents. Based on 484 accident records of a large chemical enterprise in China, the Spearman correlation coefficient was used to analyze the rationality of accident classification and the occurrence rules of accidents at different levels. In addition, this paper used TF-IDF and K-means algorithms to extract keywords and perform text clustering analysis for accidents at different levels based on accident classification. The risk factors of each accident cluster were further analyzed, and improvement measures were proposed for the sample enterprises.

RESULTS

The results show that reducing unit-level accidents can prevent group-level accidents. The accidents of the sample enterprises are mainly personal injury accidents, production accidents, environmental pollution accidents, and quality accidents. The leading causes of personal injury accidents are employees' unsafe behaviors, such as poor safety awareness, non-standard operation, illegal operation, untimely communication, etc. The leading causes of production accidents, environmental pollution accidents, and quality accidents include the unsafe state of materials, such as equipment damage, pipeline leakage, short-circuiting, excessive fluctuation of process parameters, etc. CONCLUSION: Compared with the traditional accident classification method, the accident triangle proposed in this paper based on the organizational level dramatically reduces the differences between accidents, helps enterprises quickly identify risk factors, and prevents accidents. This method can effectively prevent accidents and provide helpful guidance for the safety management of chemical enterprises.

Application of a Machine Learning Algorithm in Prediction of Abusive Head Trauma in Children

PURPOSE

We explored the application of a machine learning algorithm for the timely detection of potential abusive head trauma (AHT) using the first free-text note of an encounter and demographic information.

METHODS

First free-text physician notes and demographic information were collected for children under 5 years of age at a Level 1 Trauma Center. The control group, which included patients with head/neck injury, was compared to those with AHT diagnosed by the Child Protective Team. Differential scores accounted for words overrepresented in AHT patient vs. control notes. Sentiment scores were reflective of note positivity/negativity and subjectivity scores accounted for note subjectivity/objectivity. The composite scores reflected the patient's differential score modified by the subjectivity score. Composite, sentiment, and subjectivity scores combined with demographic information trained a Random Forest (RF) machine learning algorithm to predict AHT.

RESULTS

Final composite scores with demographic information were highly associated with AHT in a test dataset. The control group included 587 patients and the test group included 193 patients. Combining composite scores with demographic information into the RF model improved AHT classification area under the curve (AUC) from 0.68 to 0.78, with an overall accuracy of 84%. Feature importance analysis of our RF model revealed that composite score, sentiment, age, and subjectivity were the most impactful predictors of AHT. The sentiment was not significantly different between control and AHT notes (p = 0.87), while subjectivity trended higher for AHT notes (p = 0.081).

CONCLUSION

We conclude that a machine learning algorithm can recognize patterns within free-text notes and demographic information that aid in AHT detection in children.

LEVEL OF EVIDENCE

III.

Artificial Intelligence and Child Abuse and Neglect: A Systematic Review

All societies should carefully address the child abuse and neglect phenomenon due to its acute and chronic sequelae. Even if artificial intelligence (AI) implementation in this field could be helpful, the state of the art of this implementation is not known. No studies have comprehensively reviewed the types of AI models that have been developed/validated. Furthermore, no indications about the risk of bias in these studies are available. For these reasons, the authors conducted a systematic review of the PubMed database to answer the following questions: "what is the state of the art about the development and/or validation of AI predictive models useful to contrast child abuse and neglect phenomenon?"; "which is the risk of bias of the included articles?". The inclusion criteria were: articles written in English and dated from January 1985 to 31 March 2023; publications that used a medical and/or protective service dataset to develop and/or validate AI prediction models. The reviewers screened 413 articles. Among them, seven papers were included. Their analysis showed that: the types of input data were heterogeneous; artificial neural networks, convolutional neural networks, and natural language processing were used; the datasets had a median size of 2600 cases; the risk of bias was high for all studies. The results of the review pointed out that the implementation of AI in the child abuse and neglect field lagged compared to other medical fields. Furthermore, the evaluation of the risk of bias suggested that future studies should provide an appropriate choice of sample size, validation, and management of overfitting, optimism, and missing data.

Using natural language processing to identify child maltreatment in health systems

BACKGROUND

Rates of child maltreatment (CM) obtained from electronic health records are much lower than national child welfare prevalence rates indicate. There is a need to understand how CM is documented to improve reporting and surveillance.

OBJECTIVES

To examine whether using natural language processing (NLP) in outpatient chart notes can identify cases of CM not documented by ICD diagnosis code, the overlap between the coding of child maltreatment by ICD and NLP, and any differences by age, gender, or race/ethnicity.

METHODS

Outpatient chart notes of children age 0-18 years old within Kaiser Permanente Washington (KPWA) 2018-2020 were used to examine a selected set of maltreatment-related terms categorized into concept unique identifiers (CUI). Manual review of text snippets for each CUI was completed to flag for validated cases and retrain the NLP algorithm.

RESULTS

The NLP results indicated a crude rate of 1.55 % to 2.36 % (2018-2020) of notes with reference to CM. The rate of CM identified by ICD code was 3.32 per 1000 children, whereas the rate identified by NLP was 37.38 per 1000 children. The groups that increased the most in identification of maltreatment from ICD to NLP were adolescents (13-18 yrs. old), females, Native American children, and those on Medicaid. Of note, all subgroups had substantially higher rates of maltreatment when using NLP.

CONCLUSIONS

Use of NLP substantially increased the estimated number of children who have been impacted by CM. Accurately capturing this population will improve identification of vulnerable youth at high risk for mental health symptoms.

A Systematic Literature Review of the Use of Computational Text Analysis Methods in Intimate Partner Violence Research

Purpose

Computational text mining methods are proposed as a useful methodological innovation in Intimate Partner Violence (IPV) research. Text mining can offer researchers access to existing or new datasets, sourced from social media or from IPV-related organisations, that would be too large to analyse manually. This article aims to give an overview of current work applying text mining methodologies in the study of IPV, as a starting point for researchers wanting to use such methods in their own work.

Methods

This article reports the results of a systematic review of academic research using computational text mining to research IPV. A review protocol was developed according to PRISMA guidelines, and a literature search of 8 databases was conducted, identifying 22 unique studies that were included in the review.

Results

The included studies cover a wide range of methodologies and outcomes. Supervised and unsupervised approaches are represented, including rule-based classification (n = 3), traditional Machine Learning (n = 8), Deep Learning (n = 6) and topic modelling (n = 4) methods. Datasets are mostly sourced from social media (n = 15), with other data being sourced from police forces (n = 3), health or social care providers (n = 3), or litigation texts (n = 1). Evaluation methods mostly used a held-out, labelled test set, or k-fold Cross Validation, with Accuracy and F1 metrics reported. Only a few studies commented on the ethics of computational IPV research.

Conclusions

Text mining methodologies offer promising data collection and analysis techniques for IPV research. Future work in this space must consider ethical implications of computational approaches.

Diagnosis codes dramatically underestimate the burden of abuse

BACKGROUND

International Classification of Diseases (ICD) billing codes are not well-suited to estimate physical abuse prevalence among hospitalized patients and may be even less accurate in emergency departments (EDs). The Centers for Disease Control and Prevention (CDC) has recently published a child abuse and neglect syndromic surveillance definition to more accurately examine national abuse trends among ED visits.

OBJECTIVE

To retrospectively apply the CDC syndromic definition to a population of physically abused children and determine its sensitivity for abuse in an ED and at hospital discharge.

PARTICIPANTS AND SETTING

All physically abused children <5 years seen in the ED and evaluated by the child protection team from 2016 to 2020 at a large Midwestern children's hospital.

METHODS

Retrospective cross-sectional study utilizing the hospital's child protection team administrative database, the Pediatric Health Information System and the electronic health record to identify the study sample, chief complaint, and abuse-specific codes assigned in the ED and at hospital discharge. Abuse-specific codes were defined as all ICD-10-CM and Systematized Nomenclature of Medicine - Clinical Terms (SNOMED CT) codes included in the CDC syndromic definition, which was applied to the sample and its sensitivity determined.

RESULTS

Among the 550 abused patients identified, most were male (58.4 %), white (65.1 %), <2 years old (80.4 %), and had public insurance (81.6 %). When applying the CDC syndromic definition, only 11.6 % were identified as abused in the ED and 65.3 % were identified at hospital discharge.

CONCLUSIONS

The CDC syndrome surveillance definition lacks sensitivity in identifying abuse in the ED or at hospital discharge.

Child Abuse Pediatricians Assess a Low Likelihood of Abuse in Half of 2890 Physical Abuse Consults

The aim of this study was to examine the frequency with which child abuse pediatricians (CAPs) assess consultations as low versus high likelihood of abuse. In this retrospective secondary analysis of data from the Examining Siblings to Recognize Abuse (ExSTRA) study, the likelihood of abuse score for 2890 consultations at 20 medical centers was collected. Descriptive statistics were used to examine the percentage of cases representing low versus high likelihood of abuse (i.e., score of 1-4 vs. 5-7 on a 7-point scale). Linear and logistic regression analyses were used to examine score variability between medical centers. Overall, fifty-three percent of cases were assessed as low likelihood of abuse, suggesting that CAPs were equally as likely to assess a high versus low likelihood of abuse. The percentage of cases representing low likelihood of abuse differed significantly (P < .001) between medical centers after controlling for patient age, sex, race/ethnicity, twin/triplet status, injury types, and injury severity. The variability between CAP assessments at different medical centers is discussed, along with potential contributors to this variability and directions for future work.