Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Natural language processing systems for extracting information from electronic health records about activities of daily living. A systematic review

Objective

Natural language processing (NLP) can enhance research on activities of daily living (ADL) by extracting structured information from unstructured electronic health records (EHRs) notes. This review aims to give insight into the state-of-the-art, usability, and performance of NLP systems to extract information on ADL from EHRs.

Materials and Methods

A systematic review was conducted based on searches in Pubmed, Embase, Cinahl, Web of Science, and Scopus. Studies published between 2017 and 2022 were selected based on predefined eligibility criteria.

Results

The review identified 22 studies. Most studies (65%) used NLP for classifying unstructured EHR data on 1 or 2 ADL. Deep learning, combined with a ruled-based method or machine learning, was the approach most commonly used. NLP systems varied widely in terms of the pre-processing and algorithms. Common performance evaluation methods were cross-validation and train/test datasets, with F1, precision, and sensitivity as the most frequently reported evaluation metrics. Most studies reported relativity high overall scores on the evaluation metrics.

Discussion

NLP systems are valuable for the extraction of unstructured EHR data on ADL. However, comparing the performance of NLP systems is difficult due to the diversity of the studies and challenges related to the dataset, including restricted access to EHR data, inadequate documentation, lack of granularity, and small datasets.

Conclusion

This systematic review indicates that NLP is promising for deriving information on ADL from unstructured EHR notes. However, what the best-performing NLP system is, depends on characteristics of the dataset, research question, and type of ADL.

Realizing the Potential of Social Determinants Data: A Scoping Review of Approaches for Screening, Linkage, Extraction, Analysis and Interventions

Background

Social determinants of health (SDoH) like socioeconomics and neighborhoods strongly influence outcomes, yet standardized SDoH data is lacking in electronic health records (EHR), limiting research and care quality.

Methods

We searched PubMed using keywords "SDOH" and "EHR", underwent title/abstract and full-text screening. Included records were analyzed under five domains: 1) SDoH screening and assessment approaches, 2) SDoH data collection and documentation, 3) Use of natural language processing (NLP) for extracting SDoH, 4) SDoH data and health outcomes, and 5) SDoH-driven interventions.

Results

We identified 685 articles, of which 324 underwent full review. Key findings include tailored screening instruments implemented across settings, census and claims data linkage providing contextual SDoH profiles, rule-based and neural network systems extracting SDoH from notes using NLP, connections found between SDoH data and healthcare utilization/chronic disease control, and integrated care management programs executed. However, considerable variability persists across data sources, tools, and outcomes.

Discussion

Despite progress identifying patient social needs, further development of standards, predictive models, and coordinated interventions is critical to fulfill the potential of SDoH-EHR integration. Additional database searches could strengthen this scoping review. Ultimately widespread capture, analysis, and translation of multidimensional SDoH data into clinical care is essential for promoting health equity.

Natural Language Processing to Classify Caregiver Strategies Supporting Participation Among Children and Youth with Craniofacial Microsomia and Other Childhood-Onset Disabilities

Customizing participation-focused pediatric rehabilitation interventions is an important but also complex and potentially resource intensive process, which may benefit from automated and simplified steps. This research aimed at applying natural language processing to develop and identify a best performing predictive model that classifies caregiver strategies into participation-related constructs, while filtering out non-strategies. We created a dataset including 1,576 caregiver strategies obtained from 236 families of children and youth (11-17 years) with craniofacial microsomia or other childhood-onset disabilities. These strategies were annotated to four participation-related constructs and a non-strategy class. We experimented with manually created features (i.e., speech and dependency tags, predefined likely sets of words, dense lexicon features (i.e., Unified Medical Language System (UMLS) concepts)) and three classical methods (i.e., logistic regression, naïve Bayes, support vector machines (SVM)). We tested a series of binary and multinomial classification tasks applying 10-fold cross-validation on the training set (80%) to test the best performing model on the held-out test set (20%). SVM using term frequency-inverse document frequency (TF-IDF) was the best performing model for all four classification tasks, with accuracy ranging from 78.10 to 94.92% and a macro-averaged F1-score ranging from 0.58 to 0.83. Manually created features only increased model performance when filtering out non-strategies. Results suggest pipelined classification tasks (i.e., filtering out non-strategies; classification into intrinsic and extrinsic strategies; classification into participation-related constructs) for implementation into participation-focused pediatric rehabilitation interventions like Participation and Environment Measure Plus (PEM+) among caregivers who complete the Participation and Environment Measure for Children and Youth (PEM-CY).

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00149-y.

Classification of neurologic outcomes from medical notes using natural language processing

Neurologic disability level at hospital discharge is an important outcome in many clinical research studies. Outside of clinical trials, neurologic outcomes must typically be extracted by labor intensive manual review of clinical notes in the electronic health record (EHR). To overcome this challenge, we set out to develop a natural language processing (NLP) approach that automatically reads clinical notes to determine neurologic outcomes, to make it possible to conduct larger scale neurologic outcomes studies. We obtained 7314 notes from 3632 patients hospitalized at two large Boston hospitals between January 2012 and June 2020, including discharge summaries (3485), occupational therapy (1472) and physical therapy (2357) notes. Fourteen clinical experts reviewed notes to assign scores on the Glasgow Outcome Scale (GOS) with 4 classes, namely 'good recovery', 'moderate disability', 'severe disability', and 'death' and on the Modified Rankin Scale (mRS), with 7 classes, namely 'no symptoms', 'no significant disability', 'slight disability', 'moderate disability', 'moderately severe disability', 'severe disability', and 'death'. For 428 patients' notes, 2 experts scored the cases generating interrater reliability estimates for GOS and mRS. After preprocessing and extracting features from the notes, we trained a multiclass logistic regression model using LASSO regularization and 5-fold cross validation for hyperparameter tuning. The model performed well on the test set, achieving a micro average area under the receiver operating characteristic and F-score of 0.94 (95% CI 0.93-0.95) and 0.77 (0.75-0.80) for GOS, and 0.90 (0.89-0.91) and 0.59 (0.57-0.62) for mRS, respectively. Our work demonstrates that an NLP algorithm can accurately assign neurologic outcomes based on free text clinical notes. This algorithm increases the scale of research on neurological outcomes that is possible with EHR data.

Automated recognition of functioning, activity and participation in COVID-19 from electronic patient records by natural language processing: a proof- of- concept

PURPOSE

To address the feasibility, reliability and internal validity of natural language processing (NLP) for automated functional assessment of hospitalised COVID-19 patients in key International Classification of Functioning, Disability and Health (ICF) categories and levels from unstructured text in electronic health records (EHR) from a large teaching hospital.

MATERIALS AND METHODS

Eight human annotators assigned four ICF categories to relevant sentences: Emotional functions, Exercise tolerance, Walking and Moving, Work and Employment and their ICF levels (Functional Ambulation Categories for Walking and Moving, metabolic equivalents for Exercise tolerance). A linguistic neural network-based model was trained on 80% of the annotated sentences; inter-annotator agreement (IAA, Cohen's kappa), a weighted score of precision and recall (F1) and RMSE for level detection were assessed for the remaining 20%.

RESULTS

In total 4112 sentences of non-COVID-19 and 1061 of COVID-19 patients were annotated. Average IAA was 0.81; F1 scores were 0.7 for Walking and Moving and Emotional functions; RMSE for Walking and Moving (5- level scale) was 1.17 for COVID-19 patients.

CONCLUSION

Using a limited amount of annotated EHR sentences, a proof-of-concept was obtained for automated functional assessment of COVID-19 patients in ICF categories and levels. This allows for instantaneous assessment of the functional consequences of new diseases like COVID-19 for large numbers of patients.Key messagesHospitalised Covid-19 survivors may persistently suffer from low physical and mental functioning and a reduction in overall quality of life requiring appropriate and personalised rehabilitation strategies.For this, assessment of functioning within multiple domains and categories of the International Classification of Function is required, which is cumbersome using structured data.We show a proof-of-concept using Natural Language Processing techniques to automatically derive the aforementioned information from free-text notes within the Electronic Health Record of a large academic teaching hospital.

A roadmap to reduce information inequities in disability with digital health and natural language processing

People with disabilities disproportionately experience negative health outcomes. Purposeful analysis of information on all aspects of the experience of disability across individuals and populations can guide interventions to reduce health inequities in care and outcomes. Such an analysis requires more holistic information on individual function, precursors and predictors, and environmental and personal factors than is systematically collected in current practice. We identify 3 key information barriers to more equitable information: (1) a lack of information on contextual factors that affect a person's experience of function; (2) underemphasis of the patient's voice, perspective, and goals in the electronic health record; and (3) a lack of standardized locations in the electronic health record to record observations of function and context. Through analysis of rehabilitation data, we have identified ways to mitigate these barriers through the development of digital health technologies to better capture and analyze information about the experience of function. We propose 3 directions for future research on using digital health technologies, particularly natural language processing (NLP), to facilitate capturing a more holistic picture of a patient's unique experience: (1) analyzing existing information on function in free text documentation; (2) developing new NLP-driven methods to collect information on contextual factors; and (3) collecting and analyzing patient-reported descriptions of personal perceptions and goals. Multidisciplinary collaboration between rehabilitation experts and data scientists to advance these research directions will yield practical technologies to help reduce inequities and improve care for all populations.

Extracting body function information using rule-based methods: Highlighting structure and formatting challenges in clinical text

This paper describes the identification of body function (BF) mentions within the clinical text within a large, national, heterogeneous corpus to highlight structural challenges presented by the clinical text. BF in clinical documents provides information on dysfunction or impairments in the function or structure of organ systems or organs. BF mentions are embedded in highly formatted structures where the formats include implied scoping boundaries that confound existing natural language processing segmentation and document decomposition techniques. This paper describes follow-up work to adapt a rule-based system created using National Institutes of Health records to a larger, more challenging corpus of Social Security Administration data. Results of these systems provide a baseline for future work to improve document decomposition techniques.

Prediction of Future Health Care Utilization Through Note-extracted Psychosocial Factors

BACKGROUND

Persons with multimorbidity (≥2 chronic conditions) face an increased risk of poor health outcomes, especially as they age. Psychosocial factors such as social isolation, chronic stress, housing insecurity, and financial insecurity have been shown to exacerbate these outcomes, but are not routinely assessed during the clinical encounter. Our objective was to extract these concepts from chart notes using natural language processing and predict their impact on health care utilization for patients with multimorbidity.

METHODS

A cohort study to predict the 1-year likelihood of hospitalizations and emergency department visits for patients 65+ with multimorbidity with and without psychosocial factors. Psychosocial factors were extracted from narrative notes; all other covariates were extracted from electronic health record data from a large academic medical center using validated algorithms and concept sets. Logistic regression was performed to predict the likelihood of hospitalization and emergency department visit in the next year.

RESULTS

In all, 76,479 patients were eligible; the majority were White (89%), 54% were female, with mean age 73. Those with psychosocial factors were older, had higher baseline utilization, and more chronic illnesses. The 4 psychosocial factors all independently predicted future utilization (odds ratio=1.27-2.77, C -statistic=0.63). Accounting for demographics, specific conditions, and previous utilization, 3 of 4 of the extracted factors remained predictive (odds ratio=1.13-1.86) for future utilization. Compared with models with no psychosocial factors, they had improved discrimination. Individual predictions were mixed, with social isolation predicting depression and morbidity; stress predicting atherosclerotic cardiovascular disease onset; and housing insecurity predicting substance use disorder morbidity.

DISCUSSION

Psychosocial factors are known to have adverse health impacts, but are rarely measured; using natural language processing, we extracted factors that identified a higher risk segment of older adults with multimorbidity. Combining these extraction techniques with other measures of social determinants may help catalyze population health efforts to address psychosocial factors to mitigate their health impacts.

Capturing and Operationalizing Participation in Pediatric Re/Habilitation Research Using Artificial Intelligence: A Scoping Review

Background

There is increased interest in using artificial intelligence (AI) to provide participation-focused pediatric re/habilitation. Existing reviews on the use of AI in participation-focused pediatric re/habilitation focus on interventions and do not screen articles based on their definition of participation. AI-based assessments may help reduce provider burden and can support operationalization of the construct under investigation. To extend knowledge of the landscape on AI use in participation-focused pediatric re/habilitation, a scoping review on AI-based participation-focused assessments is needed.

Objective

To understand how the construct of participation is captured and operationalized in pediatric re/habilitation using AI.

Methods

We conducted a scoping review of literature published in Pubmed, PsycInfo, ERIC, CINAHL, IEEE Xplore, ACM Digital Library, ProQuest Dissertation and Theses, ACL Anthology, AAAI Digital Library, and Google Scholar. Documents were screened by 2–3 independent researchers following a systematic procedure and using the following inclusion criteria: (1) focuses on capturing participation using AI; (2) includes data on children and/or youth with a congenital or acquired disability; and (3) published in English. Data from included studies were extracted [e.g., demographics, type(s) of AI used], summarized, and sorted into categories of participation-related constructs.

Results

Twenty one out of 3,406 documents were included. Included assessment approaches mainly captured participation through annotated observations (n = 20; 95%), were administered in person (n = 17; 81%), and applied machine learning (n = 20; 95%) and computer vision (n = 13; 62%). None integrated the child or youth perspective and only one included the caregiver perspective. All assessment approaches captured behavioral involvement, and none captured emotional or cognitive involvement or attendance. Additionally, 24% (n = 5) of the assessment approaches captured participation-related constructs like activity competencies and 57% (n = 12) captured aspects not included in contemporary frameworks of participation.

Conclusions

Main gaps for future research include lack of: (1) research reporting on common demographic factors and including samples representing the population of children and youth with a congenital or acquired disability; (2) AI-based participation assessment approaches integrating the child or youth perspective; (3) remotely administered AI-based assessment approaches capturing both child or youth attendance and involvement; and (4) AI-based assessment approaches aligning with contemporary definitions of participation.

Social Determinants of Health in Physiatry: Challenges and Opportunities for Clinical Decision Making and Improving Treatment Precision

Physiatry is a medical specialty focused on improving functional outcomes in patients with a variety of medical conditions that affect the brain, spinal cord, peripheral nerves, muscles, bones, joints, ligaments, and tendons. Social determinants of health (SDH) play a key role in determining therapeutic process and patient functional outcomes. Big data and precision medicine have been used in other fields and to some extent in physiatry to predict patient outcomes, however many challenges remain. The interplay between SDH and physiatry outcomes is highly variable depending on different phases of care, and more favorable patient profiles in acute care may be less favorable in the outpatient setting. Furthermore, SDH influence which treatments or interventional procedures are accessible to the patient and thus determine outcomes. This opinion paper describes utility of existing datasets in combination with novel data such as movement, gait patterning and patient perceived outcomes could be analyzed with artificial intelligence methods to determine the best treatment plan for individual patients in order to achieve maximal functional capacity.

Linking Free Text Documentation of Functioning and Disability to the ICF With Natural Language Processing

Background: Invaluable information on patient functioning and the complex interactions that define it is recorded in free text portions of the Electronic Health Record (EHR). Leveraging this information to improve clinical decision-making and conduct research requires natural language processing (NLP) technologies to identify and organize the information recorded in clinical documentation.

Methods: We used natural language processing methods to analyze information about patient functioning recorded in two collections of clinical documents pertaining to claims for federal disability benefits from the U.S. Social Security Administration (SSA). We grounded our analysis in the International Classification of Functioning, Disability, and Health (ICF), and used the Activities and Participation domain of the ICF to classify information about functioning in three key areas: mobility, self-care, and domestic life. After annotating functional status information in our datasets through expert clinical review, we trained machine learning-based NLP models to automatically assign ICF categories to mentions of functional activity.

Results: We found that rich and diverse information on patient functioning was documented in the free text records. Annotation of 289 documents for Mobility information yielded 2,455 mentions of Mobility activities and 3,176 specific actions corresponding to 13 ICF-based categories. Annotation of 329 documents for Self-Care and Domestic Life information yielded 3,990 activity mentions and 4,665 specific actions corresponding to 16 ICF-based categories. NLP systems for automated ICF coding achieved over 80% macro-averaged F-measure on both datasets, indicating strong performance across all ICF categories used.

Conclusions: Natural language processing can help to navigate the tradeoff between flexible and expressive clinical documentation of functioning and standardizable data for comparability and learning. The ICF has practical limitations for classifying functional status information in clinical documentation but presents a valuable framework for organizing the information recorded in health records about patient functioning. This study advances the development of robust, ICF-based NLP technologies to analyze information on patient functioning and has significant implications for NLP-powered analysis of functional status information in disability benefits management, clinical care, and research.

Viewpoint: An Information Extraction Framework for Disability Determination Using a Mental Functioning Use-Case (Preprint)

Natural language processing (NLP) in health care enables transformation of complex narrative information into high value products such as clinical decision support and adverse event monitoring in real time via the electronic health record (EHR). However, information technologies for mental health have consistently lagged because of the complexity of measuring and modeling mental health and illness. The use of NLP to support management of mental health conditions is a viable topic that has not been explored in depth. This paper provides a framework for the advanced application of NLP methods to identify, extract, and organize information on mental health and functioning to inform the decision-making process applied to assessing mental health. We present a use-case related to work disability, guided by the disability determination process of the US Social Security Administration (SSA). From this perspective, the following questions must be addressed about each problem that leads to a disability benefits claim: When did the problem occur and how long has it existed? How severe is it? Does it affect the person’s ability to work? and What is the source of the evidence about the problem? Our framework includes 4 dimensions of medical information that are central to assessing disability—temporal sequence and duration, severity, context, and information source. We describe key aspects of each dimension and promising approaches for application in mental functioning. For example, to address temporality, a complete functional timeline must be created with all relevant aspects of functioning such as intermittence, persistence, and recurrence. Severity of mental health symptoms can be successfully identified and extracted on a 4-level ordinal scale from absent to severe. Some NLP work has been reported on the extraction of context for specific cases of wheelchair use in clinical settings. We discuss the links between the task of information source assessment and work on source attribution, coreference resolution, event extraction, and rule-based methods. Gaps were identified in NLP applications that directly applied to the framework and in existing relevant annotated data sets. We highlighted NLP methods with the potential for advanced application in the field of mental functioning. Findings of this work will inform the development of instruments for supporting SSA adjudicators in their disability determination process. The 4 dimensions of medical information may have relevance for a broad array of individuals and organizations responsible for assessing mental health function and ability. Further, our framework with 4 specific dimensions presents significant opportunity for the application of NLP in the realm of mental health and functioning beyond the SSA setting, and it may support the development of robust tools and methods for decision-making related to clinical care, program implementation, and other outcomes.

Translational NLP: A New Paradigm and General Principles for Natural Language Processing Research

Natural language processing (NLP) research combines the study of universal principles, through basic science, with applied science targeting specific use cases and settings. However, the process of exchange between basic NLP and applications is often assumed to emerge naturally, resulting in many innovations going unapplied and many important questions left unstudied. We describe a new paradigm of Translational NLP, which aims to structure and facilitate the processes by which basic and applied NLP research inform one another. Translational NLP thus presents a third research paradigm, focused on understanding the challenges posed by application needs and how these challenges can drive innovation in basic science and technology design. We show that many significant advances in NLP research have emerged from the intersection of basic principles with application needs, and present a conceptual framework outlining the stakeholders and key questions in translational research. Our framework provides a roadmap for developing Translational NLP as a dedicated research area, and identifies general translational principles to facilitate exchange between basic and applied research.