Development and Validation of a Natural Language Processing Tool to Identify Injuries in Infants Associated With Abuse

Routine Emergency Department Screening to Decrease Subsequent Physical Abuse

STUDY OBJECTIVE

Emergency department (ED) screening for child physical abuse has been widely implemented, with uncertain effects on child abuse identification. Our goal was to determine the effect of screening on referrals to child protective services (CPS) identifying abuse.

METHODS

We performed a retrospective cohort study of children younger than 6 years old with an ED encounter at 1 of 2 large health care systems, one of which implemented routine child abuse screening. The main outcome was initial (<2 days) or subsequent (3 to 180 days) referral to CPS identifying child abuse using linked records. We compared outcomes for the 2-year period after screening was implemented to the preperiod and nonscreening EDs using generalized estimating equations to adjust for sex, age, race/ethnicity, payor and prior ED encounters and clustered by center.

RESULTS

Of the 331,120 ED encounters, 41,589 (12.6%) occurred at screening EDs during the screening period. Screening was completed in 34,272 (82%) and was positive in 188 (0.45%). Overall, 7,623 encounters (2.3%) had a subsequent referral, of which 589 (0.2%) identified moderate or severe abuse. ED screening did not change initial (adjusted odds ratio [aOR]=1.01, 95% confidence interval [CI] 0.89 to 1.15) or subsequent referral to CPS when compared to the prescreening period (aOR=1.05, 95% CI 0.9 to 1.18) or to the nonscreening EDs (aOR=1.06, 95% CI 0.92 to 1.21).

CONCLUSION

Routine screening did not affect initial or subsequent referrals to CPS.

Developing and Testing the Usability of a Novel Child Abuse Clinical Decision Support System: Mixed Methods Study

BACKGROUND

Despite the impact of physical abuse on children, it is often underdiagnosed, especially among children evaluated in emergency departments (EDs). Electronic clinical decision support (CDS) can improve the recognition of child physical abuse.

OBJECTIVE

We aimed to develop and test the usability of a natural language processing-based child abuse CDS system, known as the Child Abuse Clinical Decision Support (CA-CDS), to alert ED clinicians about high-risk injuries suggestive of abuse in infants' charts.

METHODS

Informed by available evidence, a multidisciplinary team, including an expert in user design, developed the CA-CDS prototype that provided evidence-based recommendations for the evaluation and management of suspected child abuse when triggered by documentation of a high-risk injury. Content was customized for medical versus nursing providers and initial versus subsequent exposure to the alert. To assess the usability of and refine the CA-CDS, we interviewed 24 clinicians from 4 EDs about their interactions with the prototype. Interview transcripts were coded and analyzed using conventional content analysis.

RESULTS

Overall, 5 main categories of themes emerged from the study. CA-CDS benefits included providing an extra layer of protection, providing evidence-based recommendations, and alerting the entire clinical ED team. The user-centered, workflow-compatible design included soft-stop alert configuration, editable and automatic documentation, and attention-grabbing formatting. Recommendations for improvement included consolidating content, clearer design elements, and adding a hyperlink with additional resources. Barriers to future implementation included alert fatigue, hesitancy to change, and concerns regarding documentation. Facilitators of future implementation included stakeholder buy-in, provider education, and sharing the test characteristics. On the basis of user feedback, iterative modifications were made to the prototype.

CONCLUSIONS

With its user-centered design and evidence-based content, the CA-CDS can aid providers in the real-time recognition and evaluation of infant physical abuse and has the potential to reduce the number of missed cases.

Natural Language Processing - A Surveillance Stepping Stone to Identify Child Abuse

OBJECTIVE

We aimed to refine a natural language processing (NLP) algorithm that identified injuries associated with child abuse and identify areas in which integration into a real-time clinical decision support (CDS) tool may improve clinical care.

METHODS

We applied an NLP algorithm in "silent mode" to all emergency department (ED) provider notes between July 2021 and December 2022 (n = 353) at 1 pediatric and 8 general EDs. We refined triggers for the NLP, assessed adherence to clinical guidelines, and evaluated disparities in degree of evaluation by examining associations between demographic variables and abuse evaluation or reporting to child protective services.

RESULTS

Seventy-three cases falsely triggered the NLP, often due to errors in interpreting linguistic context. We identified common false-positive scenarios and refined the algorithm to improve NLP specificity. Adherence to recommended evaluation standards for injuries defined by nationally accepted clinical guidelines was 63%. There were significant demographic differences in evaluation and reporting based on presenting ED type, insurance status, and race and ethnicity.

CONCLUSIONS

Analysis of an NLP algorithm in "silent mode" allowed for refinement of the algorithm and highlighted areas in which real-time CDS may help ED providers identify and pursue appropriate evaluation of injuries associated with child physical abuse.

Artificial Intelligence and Child Abuse and Neglect: A Systematic Review

All societies should carefully address the child abuse and neglect phenomenon due to its acute and chronic sequelae. Even if artificial intelligence (AI) implementation in this field could be helpful, the state of the art of this implementation is not known. No studies have comprehensively reviewed the types of AI models that have been developed/validated. Furthermore, no indications about the risk of bias in these studies are available. For these reasons, the authors conducted a systematic review of the PubMed database to answer the following questions: "what is the state of the art about the development and/or validation of AI predictive models useful to contrast child abuse and neglect phenomenon?"; "which is the risk of bias of the included articles?". The inclusion criteria were: articles written in English and dated from January 1985 to 31 March 2023; publications that used a medical and/or protective service dataset to develop and/or validate AI prediction models. The reviewers screened 413 articles. Among them, seven papers were included. Their analysis showed that: the types of input data were heterogeneous; artificial neural networks, convolutional neural networks, and natural language processing were used; the datasets had a median size of 2600 cases; the risk of bias was high for all studies. The results of the review pointed out that the implementation of AI in the child abuse and neglect field lagged compared to other medical fields. Furthermore, the evaluation of the risk of bias suggested that future studies should provide an appropriate choice of sample size, validation, and management of overfitting, optimism, and missing data.